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Abstract 

In  the  current  fiscally  constrained  environment,  the  Air  Force  must  allocate 
resources  where  they  are  most  needed  and  will  be  most  effectively  used.  For  aircraft,  this 
means  spending  money  on  weapon  systems  in  a  manner  that  optimizes  aircraft 
availability  rates,  thereby  maximizing  the  warfighting  capability  of  the  Air  Force.  With 
that  in  mind,  this  thesis  endeavors  to  improve  the  analytical  capability  of  the  Air  Force  by 
demonstrating  a  definitive  link  between  operations  and  maintenance  (O&M)  spending 
and  aircraft  availability  rates.  In  order  to  do  that,  explanatory  regression  models  are 
developed  that  show  the  relationship  between  O&M  spending  and  AA  rates,  while 
controlling  for  as  many  other  significant  variables  as  the  data  allow.  Ultimately,  this 
research  was  unable  to  show  that  aircraft  availability  rates  are  significantly  influenced  by 
changes  in  O&M  spending;  however,  suggestions  for  future  research  and  potential  policy 
implications  are  discussed. 
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OPTIMIZING  AIRCRAFT  AVAILABILITY:  WHERE  TO  SPEND  YOUR  NEXT 


O&M  DOLLAR 


I:  Introduction 


Background 

In  his  leadership  message,  the  Assistant  Secretary  of  the  Air  Force  for  Financial 

Management  and  Comptroller,  Dr.  Jamie  M.  Morin  stated, 

in  this  time  of  scarce  resources  ...  every  dollar  wasted  or  inefficiently  expended  is 
an  additional  debt  passed  on  to  our  children.  ...we  are  all  charged  with 
balancing  the  imperative  of  effectively  using  resources  to  accomplish  vital 
national  goals  with  the  need  to  continuously  ...  provide  the  capabilities  needed  in 
the  wars  we  are  fighting  today,  and  prepare  for  the  uncertain  conflicts  of  the 
future  (2009). 

Regardless  of  the  dollar  value  of  the  bottom  line  Air  Force  budget,  Air  Force  decision 
makers  should  expend  resources  in  the  most  effective  manner  possible.  That  is,  use 
resources  where  they  will  have  the  most  positive  effect  on  the  mission. 

The  mission  of  the  United  States  Air  Force  is  to  “fly  fight  and  win  ...  in  air,  space 
and  cyberspace”  (Donley  and  Schwartz,  2009:3).  While  the  Air  Force  employs  many 
assets  to  achieve  this  mission,  the  most  obvious  and  direct  tools  used  are  aircraft. 

Whether  the  mission  is  close  air  support,  airlift,  aerial  refueling,  or  reconnaissance,  the 
availability  of  necessary  aircraft  to  perform  the  mission  is  a  vital  concern.  Accordingly, 
the  Air  Force  spends  a  significant  portion  of  its  budget  each  year  to  sustain  its  aircraft.  In 
fiscal  year  (FY)  2009,  the  Air  Force  spent  $42.1  billion  on  operations  and  maintenance 
(O&M)  to  support  its  people  and  equipment.  Of  those  O&M  expenditures,  the  money 
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spent  directly  on  flying  operations  totaled  $14.7  billion,  which  amounts  to  12.9%  of  the 
baseline  budget  (SAF/FMB,  2009). 

Although  the  Air  Force  allocates  a  nontrivial  portion  of  its  budget  to  support  its 
aircraft,  aircraft  availability  (AA)  rates  rise  and  fall  with  regularity.  Figure  1  shows  the 
availability  rates  of  aircraft  by  mission  type  from  FY1999  to  FY2009.  As  we  can  plainly 
see,  AA  rates  fluctuate  drastically  for  each  of  the  weapon  system  types  shown  from  year 
to  year.  Although  it  would  be  helpful  to  compare  the  historical  AA  rates  to  the  goals  set 
for  each  weapon  system  over  the  same  period,  the  Air  Force  did  not  begin  establishing 
AA  standards  until  2008.  Prior  to  that,  the  Air  Force  focused  most  of  its  attention  on 
mission  capable  (MC)  rates  and  the  major  commands  (MAJCOMs)  were  primarily 
responsible  for  setting  their  own  goals  (Tyler,  2009). 


Aircraft  Availability 


Figure  1:  Aircraft  Availability  Rates,  FY99  -  FY09  (Tirpak,  2009) 

With  that  said,  Air  Force  leaders  should  have  a  better  understanding  of  the  factors 
that  influence  AA  rates  so  that  they  are  able  to  affect  positive  change.  Without  an 
adequate  number  or  the  right  combination  of  available  aircraft,  the  Air  Force  may  not  be 
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able  to  accomplish  its  mission.  In  our  study,  we  build  on  previous  research  to  gain  a 
better  understanding  of  the  factors  that  affect  AA  rates.  Specifically,  we  show  the  affect 
that  O&M  costs  have  on  AA  rates,  which  will  allow  Air  Force  leaders  to  use  O&M 
resources  as  a  means  to  control  the  AA  rates  of  the  Air  Force  fleet. 

Purpose  of  This  Study 

In  this  study,  we  seek  to  detennine  which  factors  significantly  affect  aircraft 
availability  rates;  O&M  costs  will  be  our  primary  independent  variable  of  interest.  In 
order  to  do  that,  we  develop  explanatory  multiple  regression  models  that  show 
relationships  between  O&M  costs  and  AA  rates,  while  controlling  for  as  many  other 
significant  variables  as  our  data  allow.  We  use  these  findings  to  improve  the  analytical 
capability  of  the  Air  Force  when  trying  to  detennine  how  to  allocate  resources  so  that 
O&M  funding  may  be  used  as  a  tool  to  optimize  aircraft  availability. 

Research  Objective 

The  objective  of  this  research  is  to  develop  explanatory  models  that  demonstrate  a 
definitive  link  between  O&M  costs  and  aircraft  availability. 

Research  Questions. 

1 .  What  variables  are  significant  predictors  of  aircraft  availability  rates? 

2.  Are  aircraft  availability  rates  influenced  by  changes  in  O&M  spending? 

3.  Do  the  aircraft  availability  rates  of  some  weapon  systems  respond  to  changes 
in  O&M  costs  more  than  others? 
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4.  Can  a  single  model  be  developed  to  represent  multiple  Mission  Design  Series 
(MDS)? 

5.  Can  the  models  produced  by  this  research  be  used  as  an  effective  decision  tool 
for  the  Centralized  Asset  Management  (CAM)  office? 

Chapter  Summary 

In  this  chapter,  we  described  the  current  fiscal  environment  in  which  the  Air 
Force  must  operate.  We  outlined  the  need  for  a  robust  analytical  process  or  tool  that  can 
guide  resource  allocation  decisions  in  an  attempt  to  optimize  aircraft  availability.  Finally, 
we  outlined  the  purpose  of  this  study  and  listed  our  research  questions. 

The  rest  of  this  paper  is  structured  as  follows:  Chapter  II  provides  background 
relating  to  aircraft  availability,  a  summary  of  previous  research,  and  a  review  of  the 
resource  allocation  process.  In  Chapter  III,  we  describe  our  dataset  and  outline  the 
methods  that  will  be  used  to  analyze  our  data.  Next,  we  present  the  results  and  analysis 
in  Chapter  IV.  Finally,  we  summarize  the  results  and  provide  policy  implications  based 
on  our  findings  in  Chapter  V. 
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II:  Literature  Review 


Given  our  research  questions  and  overall  objective,  we  seek  to  expand  our 
knowledge  concerning  aircraft  availability  and  the  variables  that  may  affect  AA  rates. 
We  begin  this  chapter  by  discussing  the  importance  of  maintaining  an  adequate  quantity 
of  mission  capable  aircraft  and  provide  an  overview  of  the  metrics  used  by  the  Air  Force 
to  assess  the  health  of  its  fleet.  Next,  we  summarize  the  findings  of  previous  research 
concerning  the  factors  that  may  affect  aircraft  availability.  Then,  we  review  several 
models  that  have  been  developed  and  used  by  the  Air  Force  to  forecast  aircraft 
availability.  Finally,  we  provide  the  reader  with  an  understanding  of  both  the  old 
resource  allocation  process  and  the  new  process  developed  through  ongoing  Air  Force 
initiatives. 

Mission  Impact  of  Low  Aircraft  Availability  Rates 

As  we  stated  earlier,  the  mission  of  the  U.S.  Air  Force  is  “to  fly,  fight  and  win  . . . 
in  air,  space  and  cyberspace.”  To  achieve  that  mission,  the  Air  Force  relies  on  its  six 
“distinctive  capabilities”  which  include  the  following  (DAF,  2003): 

1 .  Air  and  Space  Superiority 

2.  Global  Attack 

3.  Rapid  Global  Mobility 

4.  Precision  Engagement 

5.  Infonnation  Superiority 

6.  Agile  Combat  Support 
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Not  surprisingly,  each  of  these  capabilities  depends  entirely  or  in  part  on  the 
availability  of  the  right  mix  of  Air  Force  aircraft.  Simply  put,  without  a  sufficient 
number  of  mission  capable  aircraft  ready  to  fly  at  any  given  time,  the  Air  Force  cannot 
perfonn  its  stated  mission. 

The  importance  of  AA  extends  to  the  unit-level  as  well.  Perhaps  the  most  direct 
costs  associated  with  inadequate  AA  are  lost  training  opportunities.  Pilots  and  their 
associated  aircrew  members  require  a  certain  number  of  sorties  and  flying  hours  per 
month  depending  on  the  aircraft  they  fly  and  the  missions  they  are  training  to  support. 
AFI  1 1-(MDS  specific  volume)  specifies  exactly  what  is  required  for  each  weapon 
system  aircrew  to  maintain  combat  mission  ready  (CMR)  status  (Lipina,  2009).  AFI  11- 
2F-16  Volume  1  defines  CMR  as  “the  minimum  training  required  for  pilots  to  be 
qualified  and  proficient  in  all  of  the  primary  missions  tasked  to  their  assigned  unit  and 
weapon  system”  (DAF,  2007:8).  In  the  case  of  F-16  aircrew,  pilots  are  required  to  fly 
nine  or  ten  sorties  per  month  depending  on  whether  they  are  experienced  or 
inexperienced.  If  this  training  requirement  cannot  be  met  because  not  enough  aircraft  are 
available,  aircrew  members  may  be  placed  on  probation  or  non-CMR  status  at  the 
discretion  of  the  squadron  commander.  If  aircrew  members  who  are  on  probation  fail  to 
meet  the  minimum  CMR  requirements  the  following  month,  they  will  be  demoted  to  non- 
CMR  status  (Lipina,  2009).  This  scenario  would  result  in  degraded  readiness  of  an  entire 
unit  and  decreased  operational  flexibility. 

Maintainers  experience  lost  training  opportunities  as  well.  When  AA  rates  are 
low,  maintainers  feel  pressure  to  fix  aircraft  quickly  using  whatever  resources  are 
available.  Often  this  means  cannibalizing  parts  from  one  aircraft  to  support  another 
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aircraft.  Cannibalizing  parts  requires  a  maintainer  to  spend  time  removing  a  part  from 
one  aircraft  and  then  installing  the  part  on  another  aircraft.  The  extra  time  involved  may 
result  in  lost  training  opportunities  for  themselves  or  a  lost  opportunity  to  train  others 
(Oliver,  2001). 

Many  of  the  other  costs  associated  with  low  AA  rates  are  so  intertwined  with  AA 
rates  themselves  that  it  is  hard  to  decouple  the  cause  and  the  effect.  For  example,  low 
AA  rates  decrease  morale  and  increase  the  workload  and  stress  for  maintainers,  which 
may  negatively  affect  retention  rates.  When  second  tenn  and  career  airmen  decide  to 
separate,  the  workload  increases  for  the  remaining  ainnen,  especially  the  5-  and  7-level 
maintainers.  Not  only  do  these  technicians  have  to  perform  more  work  on  aircraft,  but 
also  their  supervisory  and  training  responsibilities  increase  as  the  ratio  of  3 -level  airmen 
increases.  This  means  that  the  need  for  supervision  and  on-the-job  training  increases  at 
the  same  time  the  workload  increases  because  of  the  need  to  generate  aircraft  (Oliver, 
2001).  As  we  have  shown  here,  the  cost  of  low  A  A  rates  are  extensive  and  may  have 
enduring  negative  consequences. 

Fleet  Management  Metrics  Overview 

In  Air  Force  maintenance  organizations,  metrics  are  used  extensively  to  assess  the 
quality,  quantity,  and  timeliness  of  the  maintenance  actions  being  performed  as  well  as 
the  overall  health  of  the  fleet  and  even  the  readiness  of  the  personnel.  However,  when  it 
comes  to  measuring  the  health  of  the  fleet  and  the  effectiveness  of  the  maintenance 
performed,  the  two  metrics  that  dominate  are  the  AA  rate  and  the  MC  rate. 
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According  to  Air  Force  Instruction  21-101  (AFI  21-101)  Aircraft  and  Equipment 
Maintenance  Management,  the  MC  rate  is  the  percentage  of  unit-possessed  hours  that 
aircraft  are  either  fully  mission  capable  (FMC)  or  partially  mission  capable  (PMC)  for  a 
specific  period  of  measurement  (e.g.,  weekly  or  monthly).  FMC  status  simply  means  that 
an  aircraft  can  perform  all  of  its  assigned  missions.  PMC  status  means  that  an  aircraft 
can  perfonn  at  least  one,  but  not  all  of  its  assigned  missions  (DAF,  2006).  MC  rate  is 
calculated  using  equation  1 . 


MC  Rate  (%) 


FMC  Hours  +PMC  Hours 
Unit  Possessed  Hours 


x  100 


(1) 


AA,  as  defined  by  AFI  21-101,  is  the  percentage  of  a  fleet  not  in  a  depot  status  or 
not  mission  capable  (NMC)  status.  Alternatively,  the  AA  rate  is  the  percentage  of  a 
fleet’s  total  active  inventory  (TAI)  that  is  available  (mission  capable).  NMC  aircraft  are 
aircraft  that  are  unable  to  perfonn  any  of  their  wartime  missions  (DAF,  2006).  The  AA 
rate  is  calculated  using  equation  2. 


AA  Rate  (%) 


MC  Hours 
TAI  Hours 


x  100 


(2) 


Intuitively,  the  complement  of  availability  is  nonavailability,  which  consists  of 
five  components:  the  unit  possessed  not  reported  (UPNR)  rate,  the  depot  rate,  the  not 
mission  capable  maintenance  (NMCM)  rate,  the  not  mission  capable  supply  (NMCS) 
rate,  and  the  not  mission  capable  both  (NMCB)  rate  (AFLMA,  2009).  Insight  into  why 
AA  rates  are  low  may  be  garnered  from  investigating  these  five  areas  of  nonavailability, 
which  we  will  briefly  explain. 

As  the  name  states,  the  UPNR  rate  is  the  percentage  of  a  fleet’s  TAI  that  are  unit 
possessed,  but  not  reported.  When  an  aircraft  suffers  major  damage  or  is  in  need  of 
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major  maintenance,  the  owning  unit  may  be  required  to  wait  for  higher  headquarters  to 
make  a  decision  regarding  how  to  proceed.  During  this  time,  the  aircraft  would  be  UPNR 
because  the  unit  is  waiting  to  be  told  what  to  do  next.  Not  surprisingly,  the  depot  rate  is 
the  percentage  of  a  fleet’s  TAI  that  are  in  depot  status.  Typically,  these  aircraft  are  either 
awaiting  or  undergoing  depot  level  maintenance  (AFLMA,  2009).  The  NMCM  rate  is 
the  percentage  of  possessed  aircraft  that  are  unable  to  perform  primary  assigned  missions 
because  the  aircraft  is  in  need  of  maintenance  that  will  be  carried  out  by  the  unit.  The 
NMCS  rate  is  the  percentage  of  possessed  aircraft  that  are  unable  to  execute  primary 
missions  for  supply  reasons  (e.g.,  lack  of  spare  parts).  Finally,  the  NMCB  rate  is  the 
percentage  of  possessed  aircraft  that  are  unable  to  perfonn  primary  assigned  missions  for 
both  maintenance  and  supply  reasons  (DAF,  2006). 

As  stated  in  the  Air  Force  Logistics  Management  Agency’s  handbook  titled 
Maintenance  Metrics  U.S.  Air  Force  and  consistent  with  the  goals  of  Expeditionary 
Logistics  for  the  21st  Century  (eLog21),  the  AA  rate  is  the  metric  that  will  be  used  to 
measure  the  health  of  the  fleet  (AFLMA,  2009).  The  primary  reason  that  the  AA  rate  is  a 
more  useful  metric  than  the  MC  rate  is  that  it  reflects  a  more  complete  picture  of  the  fleet. 
While  the  numerator  is  the  same  for  both  metrics,  the  denominator  is  different  which 
results  in  a  gap  between  the  two  rates.  The  MC  rate  only  considers  aircraft  that  are 
possessed  by  operational  units  and  ignores  aircraft  that  are  in  UPNR  and  depot  status. 

This  means  that  the  denominator  is  a  fluid  number  and  will  always  result  in  a  rate  that  is 
greater  than  the  AA  rate.  The  AA  rate  on  the  other  hand  reflects  all  aircraft  in  the  fleet. 
Simply  put,  the  AA  rate  answers  a  question  that  is  central  to  assessing  combat  capability: 
How  many  jets  are  ready  to  fly? 
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Previous  Research  Reveals  Factors  Affecting  Aircraft  Availability 

In  our  review  of  literature  from  the  1990s  to  2009,  we  found  an  abundance  of 
research  completed  by  the  Air  Force  Institute  of  Technology  (AFIT),  the  Naval 
Postgraduate  School  (NPS),  RAND,  the  Government  Accountability  Office  (GAO),  and 
the  Air  Force  Logistics  Management  Agency  (AFLMA).  The  majority  of  these  studies 
focused  on  identifying  predictive  factors  to  be  used  in  forecasting  MC  rates.  While  these 
studies  have  contributed  a  significant  amount  to  the  existing  knowledge  on  AA,  none  of 
the  research  has  focused  directly  on  O&M  costs.  Our  investigation  seeks  to  close  this 
gap  in  knowledge. 

Although  our  research  focuses  on  AA,  the  majority  of  prior  research  discusses 
MC  rates.  The  reason  for  this  is  that  MC  rates  were  the  most  commonly  used  metric  to 
assess  fleet  health  until  2004  when  Air  Force  decision  makers  introduced  the  AA  metric 
as  a  part  of  eLog21  to  provide  an  enterprise  view  of  the  total  fleet  (Tyler,  2009).  Since 
that  time,  some  research  has  focused  on  AA  rates  and  the  factors  that  go  into  it,  but  not 
enough  for  the  purposes  of  our  literature  review.  Nevertheless,  research  regarding  MC 
rates  will  be  sufficient  since  we  know  that  total  MC  hours  are  the  most  significant  factor 
in  determining  A  A  rates. 

A  study  published  in  Air  Force  Journal  of  Logistics  in  2001  identified  53 
variables  that  may  affect  MC  rates  (Table  1).  Previous  research  and  history  have  shown 
that  these  factors  may  be  grouped  into  six  main  categories:  personnel,  environment, 
reliability  and  maintainability,  funding,  aircraft  operations,  and  logistics  operations. 
While  it  is  doubtful  that  an  entirely  complete  list  of  factors  could  be  created,  this  table 
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serves  as  a  very  good  starting  point  for  our  research.  We  will  briefly  discuss  several  of 
the  more  prevalent  factors  below. 


Table  1:  Potential  Factors  Affecting  MC  Rates  (Oliver,  et  al.,  2001) 


Personnel 

Environment 

Reliability  & 
Maintainability 

Funding 

Aircraft 

Operations 

Logistics 

Operations 

Personnel  assigned  or 
authorized 

OPSTEMPO 

factors 

TNMCM 

hours 

Replenishment 
spares  funding 

Aircraft 

utilization  rates 

TNMCS 

hours 

Personnel  in  each  skill- 
level  (1,  3,  5,7,  9  and  0) 

PERSTEMPO 

factors 

Maintenance 

downtime/reliability 

Repair 

funding 

Possessed 

hours 

Base  repair 
cycle  time 

Personnel  in  each  grade 
(E1-E9) 

Number  of 
deployments 

l—M 

General  support 

funding 

Average  sortie 
duration 

Order  and 
ship  time 

F-16  maintenance 
personnel  in  various  Air 
Force  specialty  codes 
(AFSC) 

Policy 

changes 

Code  3 
breaks 

Contractor 
logistics  support 
funding 

Flying 

hours 

Level  of 

serviceable 

inventory 

F-16  maintenance 
personnel  by  skill-level 
per  AFSC 

Contingencies 

8-hour  fix  rate 

Mission  support 
funding 

Sorties 

Level  of 

unserviceable 

inventory 

F-16  maintenance 
personnel  by  grade  per 
AFSC 

Vanishing 

Vendors 

Reparable  item 
failures 

O&M  funding 

Flying 

scheduling 

effectiveness 

Supply 

reliability 

Retention  rates  for  F-16 
maintenance  personnel 

Weather 

Cannibalization 

hours/actions 

Initial  spares 
funding 

Supply  downtime 

Personnel  per  aircraft 
ratios 

Aircraft 

age 

Repair  actions/hours 

Acquisition 
logistics  funding 

Over-Gs 

Depot  repair 
cycle  time 

Maintenance  officers 
assigned  or  authorized 

Maintenance 

man-hours 

Airframe 

hours 

Maintenance 

scheduling 

effectiveness 

Personnel 


In  his  2001  AFIT  thesis,  Captain  Steve  Oliver  used  correlation  and  regression 
analysis  to  identify  factors  associated  with  MC  rates  of  F-16C/D  aircraft.  His  results 
showed  that  as  the  number  of  inexperienced  personnel  (measured  by  rank  or  skill  level) 
increased,  MC  rates  decreased.  Higher  ratios  of  3-levels  to  either  5-  or  7-levels  (or  both) 
were  also  negatively  correlated  to  MC  rates.  Concerning  reenlistment  rates,  Oliver 
detennined  that  first  term  and  career  ainnen  along  with  the  overall  reenlistment  rate  were 
positively  correlated  with  MC  rates.  Additionally,  the  reenlistment  rate  of  eligible  crew 
chiefs  showed  a  high  degree  of  positive  correlation.  Finally,  the  ratio  of  maintainers  per 
aircraft,  and  total  maintainers  assigned  demonstrated  strong  positive  correlations  (Oliver, 
2001). 
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In  a  2003  report,  the  GAO  determined  that  shortages  of  maintenance  personnel  as 
well  as  a  lack  of  experienced  maintainers  contributed  to  low  MC  rates  (GAO,  2003). 
Similarly,  research  completed  in  2004  on  C-17  MC  rates  showed  that  crew  chief  manning 
levels  have  a  significant  positive  relationship  with  MC  rates  (Huscroft,  2004).  Finally,  an 
article  published  in  the  Air  Force  Journal  of  Logistics  in  2007  examined  F-16C/D  MC 
rates  and  found  the  percentage  of  7-  and  9-level  maintainers  to  be  significant  in 
explaining  MC  rates.  Specifically,  the  authors  developed  a  model  using  just  these  two 
dependent  variables  to  explain  82  percent  of  the  variance  observed  in  MC  rates  (Chimka 
and  Nachtmann,  2007). 

Environment 

Concerning  the  operational  environment,  Capt  Billy  Gilliland  used  regression  to 
test  the  relationship  between  13  common  measures  of  aircraft  maintenance  and  several 
dependent  variables  in  his  1990  AFIT  thesis.  Among  other  findings,  the  analysis  showed 
a  positive  correlation  between  MC  rates  and  the  average  number  of  possessed  aircraft 
(Gilliland,  1990). 

Intuitively,  aircraft  age  is  a  likely  consideration  when  discussing  availability.  The 
GAO  confirmed  this  notion  citing  aircraft  age  as  a  factor  that  affects  MC  rates. 

According  to  interviews  with  logistics  officials  from  the  services,  aircraft  failure  rates 
follow  a  curve  that  is  similar  to  the  “bathtub  curve”  depicted  in  Figure  2.  While  the  Air 
Force’s  inventory  continues  to  age,  failures  become  more  common  as  aircraft  reach  the 
end  of  their  useful  life.  Exacerbating  this  effect  are  increased  deployments  over  recent 
years,  which  has  forced  aircraft  to  operate  at  higher  than  nonnal  rates  and  has  accelerated 
aging  concerns  (GAO,  2003). 
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Figure  2:  Bathtub  Curve  (Wilkins,  2002) 

Prior  to  2002,  the  Air  Force  structured  its  flying  units  under  what  was  called  an 
“Objective  Wing  Structure.”  Maintenance  organizations  reported  to  either  the  operations 
group  commander  or  the  logistics  group  commander.  In  October  of  2002,  the  Air  Force 
transitioned  to  the  “Combat  Wing  Structure,”  which  aligned  all  maintenance  units  under 
a  maintenance  group  commander  with  the  goals  of  “enhancing  core  competencies, 
improving  aircraft  sortie  production,  and  improving  fleet  health”  (Barthol,  2005:3). 
Research  completed  in  2005  concluded  that  this  organizational  change  was  effective  in 
meeting  its  stated  goals  (Barthol,  2005). 

Reliability  &  Maintainability 

Gilliland  showed  a  negative  correlation  between  MC  rates  and  both  the 
cannibalization  rate  and  awaiting  maintenance  discrepancies  (Gilliland,  1990).  Next, 
Lieutenant  Commander  Patricia  Moore  showed  that  cannibalizations  are  negatively 
correlated  with  FMC  rates  of  deployed  aircraft  (Moore,  1998).  Lastly,  Oliver’s  research 
found  a  strong  positive  correlation  linking  8-hour  fix  rates  and  MC  rates  (Oliver,  2001). 
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Funding 

According  to  a  RAND  study  completed  in  2009,  aircraft  maintained  by  contractor 
logistics  support  (CLS)  have  a  higher  proportion  of  contractually  fixed  costs  each  year 
than  organically  supported  aircraft.  RAND  states  that  as  a  result  “CLS  programs  have 
been  less  affected  by  funding  instability  than  have  organically  supported  programs,  which 
must  often  reduce  funding  for  spare  parts  when  budgets  are  cut”  (Boito,  2009:46).  To 
illustrate  this  point,  RAND  compared  the  total  not  mission  capable  supply  (TNMCS) 
achieved  rates  and  standards  between  CLS  and  organically  supported  aircraft  with  the 
same  mission  over  a  three-year  period.  Figure  3  illustrates  a  representative  sample  of 
their  findings  (Boito,  2009). 


Figure  3:  Comparison  of  TNMCS  Rates  and  Standards  for  CLS  and  Organically  Supported  Trainer  Aircraft 

(Boito,  2009) 

While  all  aircraft  exceeded  their  respective  standard,  CLS  aircraft  are  held  to  a 


much  tougher  standard  than  organic  aircraft.  RAND  argues  that  CLS  aircraft  achieve 


better  (i.e.,  lower)  TNMCS  rates  because  they  receive  more  funding  than  organic  aircraft. 
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RAND  further  concludes  that  high  AA  rates  are  “largely  a  function  of  the  resources 
devoted  to  maintain  them”  (Boito,  2009:46). 

Additionally,  the  GAO  reports  that  officials  from  all  the  services  blame 
underfunding  spare  parts  inventories,  maintenance  depots,  and  other  areas  of  maintenance 
and  supply  as  a  reason  for  low  MC  rates  (GAO,  2003). 

Aircraft  Operations 

Moore’s  analysis  found  that  an  increase  in  the  number  of  sorties  causes  FMC 
rates  to  increase;  however,  an  increase  in  the  number  of  sorties  combined  with  an 
increase  in  the  number  of  cannibalizations  causes  FMC  rates  to  significantly  decrease 
(Moore,  1998). 

Logistics  Operations 

Pertaining  to  logistics,  Gilliland  determined  that  awaiting  parts  discrepancies  are 
negatively  correlated  with  MC  rates  (Gilliland,  1990).  Moore’s  research  found  that  FMC 
rates  increase  as  the  percentage  of  requests  for  consumable  or  repairable  items  that  are 
filled  in  one  to  two  days  increases  (Moore,  1998).  Oliver’s  research  found  that  the  most 
significant  correlations  between  logistics  variables  and  MC  rates  appeared  with  a  lag  of 
two  quarters;  however,  the  statistical  significance  was  not  strong  enough  to  warrant 
inclusion  in  his  models  (Oliver,  2001).  Lastly,  the  GAO  reports  that  shortages  of  spare 
parts  contribute  to  low  MC  rates.  They  say  that  this  may  be  caused  by  underestimates  of 
demand,  contracting  issues,  or  other  problems  (GAO,  2003).  We  summarize  the  findings 
of  previous  research  in  Table  2. 
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Table  2:  Variable  Correlations  with  Aircraft  Availability  Rates 


Category 

Variable 

Correlation 

Author 

Personnel 

Ratio  of  3 -levels  to  5 -levels 

Negative 

Oliver,  2001 

Ratio  of  3 -levels  to  7-levels 

Negative 

Oliver,  2001 

Total  #  of  Inexperienced 
Maintainers  by  Rank  or  Skill 
Level 

Negative 

Oliver,  2001 

Maintainers  Per  Aircraft 

Positive 

Oliver,  2001 

Total  #  of  Maintainers 

Positive 

Oliver,  2001 

Overall  Reenlistment  Rate 

Positive 

Oliver,  2001 

Reenlistment  Rate  of  First- 
Term  Airmen 

Positive 

Oliver,  2001 

Reenlistment  Rate  of  Career 
Airmen 

Positive 

Oliver,  2001 

Reenlistment  Rate  of  Eligible 
Crew  Chiefs 

Positive 

Oliver,  2001 

Crew  Chief  Manning  Levels 

Positive 

Huscroft,  2004 

Percentage  of  7-level 
Maintainers 

Positive 

Chimka  and 
Nachtmann,  2007 

Percentage  of  9-level 
Maintainers 

Positive 

Chimka  and 
Nachtmann,  2007 

Environment 

Average  #  of  Possessed 
Aircraft 

Positive 

Gilliland,  1990 

Aircraft  Age 

Mixed 

(Bathtub 

Curve) 

GAO,  2003 

Transition  to  Combat  Wing 
Structure  in  2002 

Positive 

Barthol,  2005 

Reliability  & 
Maintainability 

Cannibalization  Rate 

Negative 

Gilliland,  1990; 
Moore,  1998; 
Oliver,  2001 

Awaiting  Maintenance 
Discrepancies 

Negative 

Gilliland,  1990 

8 -Hour  Fix  Rate 

Positive 

Oliver,  2001 

Funding 

CLS  supported 

Positive 

RAND,  2009 

Aircraft  Operations 

Sorties 

Mixed 

Moore,  1998; 

Logistics  Operations 

Awaiting  Parts  Discrepancies 

Negative 

Gilliland,  1990 

%  of  Requests  for 
Consumables  Filled  in  1-2 
days 

Positive 

Moore,  1998 
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Establishing  Aircraft  Availability  Goals 

According  to  AFI  21-103,  Equipment  Inventory,  Status  and  Utilization  Reporting, 
“MAJCOMs  establish  capability  goals  in  coordination  with  the  Air  Staff  to  include  but 
not  limited  to  MC,  total  not  mission  capable  maintenance  (TNMCM),  and  TNMCS. 

These  goals  enable  HQ  USAF  to  assess  resource  allocation  funding  on  a  quarterly  basis” 
(DAF,  2005:9).  Although  these  lines  are  taken  from  the  most  current  version  of  AFI  21- 
103,  the  infonnation  is  outdated.  Since  implementation  of  eLog21  (which  we  will 
discuss  in  detail  later  in  this  chapter),  the  MAJCOMs  no  longer  set  their  own  capability 
goals.  Instead,  the  Air  Force  Directorate  for  Logistics,  Installations  &  Mission  Support 
(AF/A4/7)  sets  common  capability  standards  for  each  weapon  system  (i.e.,  MDS)  across 
active  duty  units  and  a  complementary  set  of  standards  for  guard  and  reserve  units  (Tyler, 
2009). 

In  2007,  the  Chief  of  Staff  of  the  Air  Force— Weapon  System  Review  directed 
emphasis  on  AA  instead  of  MC  rates  when  assessing  fleet  health  (Tyler,  2009).  Shortly 
thereafter,  AF/A4/7  developed  a  methodology  for  detennining  AA  standards,  given  in 
equation  3. 

PAI 

AA  Standard  (%)  =  —  x  MC  Standard  (3) 

In  this  equation,  primary  aircraft  inventory  (PAI)  is  the  number  of  aircraft 
assigned  to  a  unit  for  the  performance  of  its  operational  mission.  The  MC  standard  is 
based  on  the  summation  of  MC  hours  required  for  all  units  to  meet  their  operational, 
training,  and  test  requirements  (HQ  AFMC/A4,  2009). 

Since  this  equation  is  tied  directly  to  MC  rate  standards,  the  value-added  from  this 
new  metric  is  uncertain.  Currently,  AF/A4  is  working  on  developing  a  new  methodology 
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with  the  goal  of  directly  linking  AA  standards  to  readiness  and  decoupling  AA  from  MC 
standards  (Tyler,  2009). 

Aircraft  Sustainability  Models  and  Aircraft  Availability  Forecasting  Models 

Over  the  years,  the  Air  Force  has  used  many  different  models  to  forecast  MC  and 
AA  rates  as  well  as  the  resources  required  to  support  its  weapon  systems.  Although 
many  of  the  models  have  been  proven  to  provide  useful  results,  there  is  currently  no 
approved  Air  Force  method  to  forecast  MC  or  AA  rates  (OSD,  2009).  We  examine 
several  of  the  prevailing  models  in  order  to  gain  an  understanding  of  the  techniques  and 
variables  that  are  used,  as  well  as  to  see  what  role,  if  any,  O&M  costs  have  played. 

Logistics  Composite  Model 

Created  in  the  late  1960s,  the  Logistics  Composite  Model  (LCOM)  is  a 
“stochastic,  discrete-event  simulation  that  relies  on  probabilities  and  random  number 
generators  to  model  scenarios  in  a  maintenance  unit  by  manipulating  certain  variables” 
(Cole  et  ah,  2007: 1).  Although  the  LCOM  can  calculate  the  resources  required  to  support 
a  weapon  system  at  a  given  capability  level  (defined  as  sortie  generation)  considering  a 
variety  of  variables,  it  is  most  prominently  used  by  the  Air  Force  to  detennine  manpower 
levels  in  operational  maintenance  units.  Specifically,  the  LCOM  is  used  by  the 
MAJCOMs  to  establish  65-70%  of  their  maintenance  manpower  requirements  (Cole  et 
ah,  2007).  On  the  other  hand,  the  LCOM  does  not  directly  consider  the  O&M  funds 
needed  to  support  a  weapon  system. 
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Aircraft  Sustainability  Model 

The  Aircraft  Sustainability  Model  (ASM)  is  used  to  detennine  the  number  of 
spare  parts  required  at  Air  Force  bases  and  depots  during  wartime  operations,  peacetime 
operations,  or  combined  operations.  Given  a  desired  level  of  aircraft  availability  or  other 
readiness  measure,  the  ASM  specifies  the  exact  quantity  and  optimal  mix  of  spare  parts 
in  order  to  meet  that  goal.  Logistics  planners  currently  use  the  ASM  for  base-level 
applications  such  as  calculating  the  spare  parts  needed  to  sustain  a  squadron  of  F- 15s 
during  a  60-day  deployment  in  order  to  achieve  an  80  percent  AA  rating  at  the  end  of  day 
60  (Blazer  and  Sloan,  2007).  From  our  discussions  with  analysts  currently  working  in  the 
Air  Force  Materiel  Command  Cost  Analysis  (AFMC/FMC)  office  and  the  CAM  office, 
we  understand  that  this  model  does  not  inform  enterprise  level  resource  allocation 
decisions. 

Mobility  Aircraft  Availability  Forecasting  Simulation  Model 

Beginning  in  2003  and  continuing  through  at  least  2005,  contractors  from 
Northrop  Grumman  and  Wright  State  University  developed  the  Mobility  Aircraft 
Availability  Forecasting  Simulation  Model  (MAAF)  in  response  to  the  Air  Mobility 
Command  (AMC)  Directorate  of  Logistics’  request  for  a  better  forecasting  tool.  MAAF 
is  an  object-oriented  modeling  and  simulation  tool  that  is  purportedly  capable  of 
predicting  AA  rates,  providing  “what  if’  analysis,  and  offering  insight  into  problems  that 
may  affect  AA  (Wall,  2004;  Ciarallo  et  ah,  2005).  Although  the  model  proved  to  be  a 
useful  prototype  in  laboratory  conditions,  AMC  detennined  the  model  was  not  ready  for 
implementation  in  real-world  operations. 
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Funding/Availability  Multimethod  Allocator  for  Spares 

As  recently  as  2001,  the  Air  Forced  utilized  the  Funding/ Availability 
Multimethod  Allocator  for  Spares  (FAMMAS)  model  to  forecast  MC  rates  for  each 
mission  design  series  (MDS)  in  its  inventory.  Employing  time-series  forecasting 
methods,  FAMMAS  uses  the  last  three  years  of  historical  TNMCS  and  TNMCM  data 
combined  with  past,  present,  and  future  spares  funding  to  forecast  MC  rates.  While  it 
produces  useful  results,  time-series  models  like  FAMMAS  do  not  provide  insight  into 
potential  cause-and-effect  relationships  that  may  be  exploited  to  affect  MC  rates. 
FAMMAS  produces  its  forecasts  by  simply  projecting  data  trends,  not  by  using 
explanatory  models.  Furthennore,  FAMMAS  does  not  incorporate  any  operations, 
personnel,  or  environment-related  variables  in  the  model;  it  uses  only  TNMCS  and 
TNMCM  data  that  act  as  adjustment  factors.  Consequently,  FAMMAS  is  not  an 
effective  tool  to  use  for  policy  or  resource  decisions  because  of  the  limited  scope  of 
variables  used  in  the  model  and  because  the  relationships  between  the  variables  are 
largely  unknown  (Oliver,  et  al.,  2001). 

Aircraft  Availability  Model 

Introduced  as  part  of  the  Secondary  Item  Requirements  System  (D041  then,  now 
D200A)  in  the  late  1980s,  the  Aircraft  Availability  Model  (AAM)  is  a  tool  that 
maximizes  aircraft  availability  given  some  level  of  funding.  Using  marginal  analysis,  the 
AAM  is  able  to  build  AA  curves  (see  Figure  4)  which  can  be  used  to  prioritize  funding 
for  a  given  set  of  weapon  systems  (Blazer  and  Sloan,  2007).  However,  the  AAM 
considers  an  aircraft  available  if  it  is  not  awaiting  resupply  of  a  spare  part.  This  means 
that  the  model  is  only  concerned  with  minimizing  the  TNMCS  rate  given  some  level  of 
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funding;  it  does  not  account  for  the  other  aspect  of  non-availability  such  as  the  NMCM, 
depot,  or  UPNR  rates.  The  Air  Force  logistics  community  uses  the  model  to  compute  the 
quantities  needed  for  safety  levels  of  the  spare  parts  that  it  manages.  The  resulting  safety 
levels  become  part  of  the  overall  requirement  that  drives  budget  requests,  repair  planning, 
and  spare  parts  purchases  (Hill,  2007). 


Figure  4:  Aircraft  Availability  Curve  (Blazer  and  Sloan,  2007) 


Weapon  System  Sustainment  Resource  Allocation  Process  Prior  to  Centralized 
Asset  Management 

Prior  to  FY2008,  the  Air  Force  replicated  the  process  to  determine  weapon  system 
sustainment  requirements,  allocate  resources,  and  execute  funds  across  each  of  the  ten 
MAJCOMS  (including  the  Guard  and  Reserves)  through  stove-piped  business  areas. 
Figure  5  approximately  represents  this  process.  Requirements  determination  began  with 
the  MAJCOMS  developing  their  individual  requirements  with  input  from  AFMC  product 
and  logistics  centers.  Then,  each  MAJCOM  created  their  budget  and  program  objective 
memorandum  (POM)  inputs  based  on  those  requirements  and  submitted  them  to  Air  Staff 
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(Naguy  and  Keck,  2007).  At  this  stage,  requirements  usually  exceeded  the  resources 
available  so  resources  were  allocated  on  a  “percent  funded”  basis  (McKown,  2009). 
After  enactment  of  funds,  Air  Staff  sent  funds  to  the  MAJCOMS  for  execution.  Finally, 
the  MAJCOMS  provided  funds  to  the  appropriate  AFMC  product  and  logistics  centers 
for  every  program  they  operated  on  an  expense-by-expense  basis  for  execution. 
Additionally,  product  and  logistics  centers,  depots,  and  supply  operations  exchanged 
funds  within  AFMC.  As  a  result,  over  two  million  transactions  occurred  every  year 
between  AFMC’s  supply  and  maintenance  activities  alone  (Naguy  and  Keck,  2007). 


Product  Ctr 
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Product  Ctr 


Requirements 


Programming  /  Budget  Executii 


Depot 
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Resources  Requirements^  Q  checkbooks 


Figure  5:  Pre-FY2008  Requirements  Determination,  Resource  Allocation,  &  Execution  Process 

(Naguy  and  Keck,  2007) 

This  process  resulted  in  numerous  inefficiencies,  shortcomings,  and  unfavorable 
outcomes.  Every  command  devoted  significant  time,  money,  and  manpower  into  parallel 
activities.  Since  the  process  started  at  low  organizational  levels,  the  lead-time  for 
formulating  requirements  was  pushed  well  ahead  of  the  execution  of  funds,  which  limited 
the  flexibility  of  the  entire  process  to  respond  to  changing  conditions.  Next,  because  each 
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of  the  commands  was  concerned  with  getting  their  fair  share  of  the  available  sustainment 
funds,  the  process  encouraged  an  adversarial  relationship  between  the  operating 
MAJCOMS,  Air  Staff  and  AFMC.  Furthermore,  since  resources  were  traditionally 
allocated  on  a  “percent  funded”  basis,  MAJCOMs  had  an  incentive  to  artificially  inflate 
their  actual  requirements  so  that  they  might  avoid  receiving  only  a  percentage  of  what 
they  had  requested  (Naguy  and  Keck,  2007).  Additionally,  this  practice  of  unconstrained 
requirements  determination  was  left  virtually  unchecked  because  resource  allocation  was 
not  based  on  performance  (McKown,  2009). 

Fleet  management  was  possibly  the  biggest  shortcoming  of  the  old  resource 
allocation  process.  In  many  cases,  more  than  one  MAJCOM  operates  a  particular 
weapon  system.  As  an  example,  six  MAJCOMs  currently  fly  the  F-15.  Under  the  old 
process,  six  operating  MAJCOMs  determined  their  requirements  for  their  share  of  the  F- 
1 5  fleet,  but  no  single  organization  or  individual  was  responsible  for  the  resources 
necessary  to  support  the  fleet  as  a  whole.  As  a  result,  one  weapon  system  was  “owned” 
by  six  separate  entities,  but  no  single  entity  had  the  scope  or  authority  necessary  to 
manage  the  entire  fleet  from  a  holistic  perspective  (Naguy  and  Keck,  2007). 

Finally,  due  to  the  different  procedures  used  and  subsequent  inconsistencies 
inherent  in  the  requirements  detennination  and  resource  allocation  process,  there  was  not 
a  feasible  way  to  detennine  the  impact  of  funding  reductions  on  aircraft  availability.  This 
shortcoming  meant  that  Air  Force  leaders  were  unable  to  know  if  the  needs  of  the 
warfighter  were  going  to  be  met  in  an  environment  of  constrained  resources  (McKown, 
2009). 
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Expeditionary  Logistics  for  the  21st  Century  and  Centralized  Asset  Management 

Given  the  myriad  shortcomings  of  the  status  quo,  the  Air  Force  needed  to  find  a 
better  way  of  doing  business.  As  a  result  of  direction  from  Air  Force  leaders  and 
consistent  with  the  ubiquitous  and  overarching  Air  Force  Smart  Operations  for  the  21st 
Century  (AFS021)  effort,  the  Air  Force  Logistics  community  developed  eLog21  as  a 
strategic  action  plan  that  seeks  to  “fundamentally  change  the  way  logistics  is 
accomplished  Air  Force  wide”  (eLog21  Fact  Sheet,  2009).  According  to  the  eLog21  fact 
sheet,  the  campaign  is  composed  of  a  number  of  initiatives  and  ultimately  strives  to  reach 
two  goals:  increase  equipment  availability  to  match  aircraft  availability  (AA)  targets,  and 
reduce  operations  and  support  (O&S)  costs  by  10%  (eLog21  Fact  Sheet,  2009). 

CAM  is  a  specific  eLog21  initiative  undertaken  jointly  by  AF/A4P,  SAF/FMB 
and  AFMC  whose  mission  is  to  “centralize  and  integrate  management  of  Air  Force 
sustainment  to  optimize  warfighting  capability  through  effective  and  efficient  allocation 
of  resources  across  the  enterprise”  (Naguy  and  Keck,  2007:5).  To  achieve  this  mission, 
CAM  centralizes  programming,  budgeting,  and  execution  of  weapon  system  resources 
within  AFMC  while  standardizing  and  streamlining  requirements  determination  for  the 
Active  Duty  Air  Force;  currently,  CAM  does  not  manage  weapon  systems  operating  in 
Air  Force  Reserve  Command  (AFRC)  or  the  Air  National  Guard  (ANG).  As  a  result, 
CAM  provides  the  Air  Force  with  an  enterprise  level  view  of  its  fleet,  which  makes  it 
possible  to  maximize  warfigthing  capability  through  the  optimization  of  aircraft 
availability  (Naguy  and  Keck,  2007).  In  a  nutshell,  CAM  was  created  to  fix  the 
burdensome  requirements  determination,  resource  allocation  and  execution  process 
described  in  the  previous  section. 
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Beginning  in  fiscal  year  2008,  CAM  implemented  the  new  process  depicted  in 
Figure  6.  Requirements  determination  begins  with  the  lead  MAJCOMs  for  each 
particular  weapon  system  working  with  the  additional  user  MAJCOMs  to  fonnulate  total 
system  requirements.  Then,  the  lead  MAJCOMs  collaborate  with  AFMC  product  and 
logistics  centers  to  finish  developing  and  prioritizing  requirements  from  an  Air  Force 
enterprise  perspective.  Once  completed,  AFMC  submits  a  consolidated  POM  and  budget 
request  to  Air  Staff.  Following  enactment  of  funds,  AFMC  provides  money  directly  to 
the  appropriate  product  centers  and  logistics  centers  for  execution.  Finally,  CAM  has  the 
ability  to  manage  sustainment  resource  trade  space  throughout  the  year  of  execution 
because  there  are  no  longer  multiple  “owners”  and  multiple  “checkbooks”  being 
maintained  by  the  MAJCOMs;  resources  are  now  holistically  managed  by  the  CAM 
office  (Naguy  and  Keck,  2007). 


Requirements 

Determination 


REPRESENTATIVE  DEPICTION 

Proarammina/Budaet  Execution 


RESULTS 


Figure  6:  New  Centralized  Asset  Management  Process  (Naguy  and  Keck,  2007) 
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We  are  interested  in  the  new  CAM  process  because  it  lays  the  foundation  to 
provide  the  Air  Force  with  the  organizational  structure  necessary  to  holistically  manage 
weapon  systems.  In  other  words,  given  this  new  process,  the  Air  Force  will  be  able  to 
use  O&M  funding  as  a  tool  to  optimize  aircraft  availability.  However,  the  Air  Force 
must  use  the  correct  metrics  to  measure  weapon  system  availability  and  it  must  employ 
robust  analytical  tools  to  guide  resource  allocation  decisions.  We  argue  that  the  Air 
Force  is  using  the  correct  metrics  to  measure  weapon  system  availability;  however,  it 
does  not  currently  have  robust  analytical  tools  or  processes  in  place  to  guide  its  resource 
allocation  decisions.  Our  research  seeks  to  establish  a  definitive  link  between  O&M 
costs  and  AA  so  that  decision  makers  will  have  the  information  they  need  to  optimize 
AA. 

Chapter  Summary 

In  previous  sections,  we  discussed  the  importance  of  maintaining  an  adequate 
quantity  of  mission  capable  aircraft  and  provided  an  overview  of  the  metrics  used  by  the 
Air  Force  to  assess  the  health  of  its  fleet.  We  summarized  the  findings  of  previous 
research  concerning  that  factors  that  may  affect  aircraft  availability.  Then,  we  provided 
an  explanation  of  how  the  Air  Force  establishes  AA  rate  standards.  Next,  we  reviewed 
several  models  that  have  been  developed  and  used  by  the  Air  Force  to  forecast  AA  rates. 
Finally,  we  detailed  how  the  Air  Force  previously  determined  weapon  system 
sustainment  requirements,  allocated  resources,  and  executed  funds.  We  explained  how 
eLog21  and  its  subsequent  initiative  Centralized  Asset  Management  have  provided  the 
organizational  change  necessary  to  allow  the  Air  Force  to  manage  its  sustainment 
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resources  holistically.  Despite  the  significant  amount  of  research  already  accomplished, 
we  find  a  significant  gap  concerning  the  relationship  between  O&M  costs  and  AA,  and 
thus  see  the  need  for  further  analysis. 
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Ill:  Data  Collection  and  Methodology 


As  we  have  shown  in  the  literature  review,  a  multitude  of  factors  influence  AA 
rates.  Due  to  the  complexity  and  numerous  relationships  that  are  possible  among  the 
factors,  we  collected  data  for  an  assortment  of  variables  in  order  to  build  a  dataset 
sufficient  for  constructing  explanatory  models.  First,  we  explain  the  scope  of  our  data 
collection  and  research.  Second,  we  acknowledge  the  sources  that  we  used  to  obtain  the 
data.  Third,  we  describe  each  of  the  variables  while  discussing  the  limitations  within  the 
dataset.  Next,  we  describe  how  we  analyze  the  variables  in  order  to  gain  a  better 
understanding  of  their  characteristics  and  predictive  ability.  Finally,  we  discuss  the 
methods  that  we  use  to  build  our  explanatory  models  and  generate  results. 

Scope  of  Data  Collection  and  Research 

The  availability  and  reliability  of  data,  specifically  the  data  needed  to  capture 
O&M  costs,  determines  the  scope  of  our  research.  Much  of  the  knowledge  we  rely  on  to 
make  decisions  regarding  our  cost  data  come  from  interviews  with  contractors  who 
maintain  the  Air  Force  Total  Ownership  Cost  (AFTOC)  database  and  an  AFTOC  users’ 
training  workshop. 

As  we  discussed  in  Chapter  II,  the  scope  of  CAM’s  mission  extends  only  to  the 
Active  Duty  Air  Force;  it  does  not  manage  the  O&M  funds  for  weapon  systems  that 
operate  in  AFRC  or  ANG.  For  this  reason,  we  only  analyze  Active  Duty  aircraft  in  our 
study  because  our  ultimate  goal  is  to  advance  the  analytical  capability  of  CAM  by 
demonstrating  a  definitive  link  between  O&M  spending  and  AA.  Additionally,  research 
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and  development  appropriations  fund  the  majority  of  weapon  systems  operated  by  Air 
Force  Materiel  Command  (AFMC),  not  O&M  appropriations.  Thus,  we  will  not  include 
any  data  attributable  to  aircraft  assigned  within  AFMC. 

At  the  beginning  of  FY1998,  the  Air  Force  made  significant  changes  to 
accounting  classifications,  particularly  those  codes  that  capture  costs  related  to  flying 
operations.  Therefore,  our  data  collection  begins  with  FY1998.  The  data  maintained  in 
AFTOC  are  subject  to  updates  on  a  recurring  basis  as  new  information  becomes  available 
and  corrections  are  made.  Because  of  this,  our  last  period  of  data  is  for  the  fourth  quarter 
of  FY08.  We  reason  that  data  from  this  period  should  be  static  and  no  longer  subject  to 
significant  updates  or  corrections.  Finally,  in  an  attempt  to  keep  our  definition  of  O&M 
costs  standard  across  all  MDS,  we  only  evaluate  organically  maintained  aircraft  in  our 
study.  CLS  maintained  aircraft  report  their  costs  to  different  accounting  classifications 
than  organically  maintained  aircraft  so  the  costs  are  not  directly  comparable. 

Since  we  will  analyze  only  Active  Duty  (excluding  AFMC),  organically 
maintained  aircraft  with  data  from  1998  -  2008,  we  can  further  narrow  the  scope  of  our 
research  to  MDS  that  fit  this  criteria.  Specifically,  we  choose  to  analyze  aircraft  that 
have  data  available  from  1998  -  2008,  that  are  still  in  the  active  Air  Force  inventory,  and 
that  have  a  TAI  of  at  least  19.  Table  3  lists  the  MDS  that  we  analyze  in  this  study. 


Table  3:  List  of  MDS  Chosen  for  Study 


Ml 

DS 

A-10A 

F-15E 

OA-10A 

F-16C 

B-1B 

F-16D 

B-2A 

KC-135R 

B-52H 

KC-135T 

F-15C 

T-38A 

F-15D 

T-38C 

29 


Data  Sources  and  Variables 


Data  Sources 

We  obtained  data  for  this  study  using  three  databases:  the  Logistics  Installations 
and  Mission  Support  -  Enterprise  View  (LIMS-EV)  database,  the  AFTOC  management 
information  system,  and  the  Reliability  and  Maintainability  Information  System 
(REMIS).  Created  as  an  eLog21  initiative,  LIMS-EV  provides  a  single  point  of  entry  to  a 
variety  of  legacy  data  systems  such  as  REMIS  and  the  Multi-echelon  Resources  and 
Logistics  Network  (MERLIN).  LIMS-EV  allows  users  to  acquire  standardized  data  and 
tracks  metrics,  trends,  and  results  (LIMS-EV  Fact  Sheet,  2010).  We  found  the  AA, 
aircraft  age,  TAI,  and  cannibalization  data  using  LIMS-EV.  AFTOC  is  a  tool  that 
integrates  cost,  logistic,  and  personnel  data  from  more  than  a  dozen  legacy  systems  into  a 
single  fonnat.  In  order  to  present  useful,  coherent  data,  AFTOC  assigns  (or  allocates 
when  required)  the  data  to  weapon  systems,  bases,  and  MAJCOMs  according  to  standard 
business  rules  (AFTOC,  2009).  Our  O&M  cost  and  personnel  data  came  from  the 
AFTOC  database.  REMIS  is  the  Air  Force’s  primary  database  for  aircraft  usage  data. 
Similar  to  LIMS-EV  and  AFTOC,  REMIS  interfaces  with  a  variety  of  other  systems  to 
provide  consolidated  data  (Oliver,  2001).  We  used  REMIS  to  retrieve  our  data  for  flying 
hours,  sorties,  and  landings. 

Dependent  Variable:  Aircraft  Availability 

Our  goal  in  this  study  is  to  develop  explanatory  models  that  will  demonstrate  a 
definitive  link  between  O&M  costs  and  AA.  Thus,  AA  will  serve  as  our  dependent 
variable.  We  retrieved  our  data  from  LIMS-EV  at  the  MDS  level,  by  MAJCOM,  by 
fiscal  year  quarter.  AA  is  most  commonly  expressed  as  a  rate;  however,  we  also  obtained 
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AA  data  in  the  fonn  of  the  average  number  of  available  aircraft  and  the  total  number  of 
available  hours.  Table  4  shows  an  example  of  this  data. 


Table  4:  Subset  of  Normalized  Aircraft  Availability  Data  from  LIMS-EV 


FY 

Q 

MAJCOM 

MDS 

AA  Rate 

AA# 

AA  Hrs 

1998 

Q1 

ACC 

A- 10 

0.725 

117 

259,244 

1998 

Q2 

ACC 

A- 10 

0.671 

109 

234,876 

1998 

Q3 

ACC 

A- 10 

0.692 

112 

243,984 

1998 

Q4 

ACC 

A- 10 

0.703 

113 

249,375 

1999 

Q1 

ACC 

A- 10 

0.705 

108 

237,482 

1999 

Q2 

ACC 

A- 10 

0.668 

101 

218,402 

1999 

Q3 

ACC 

A- 10 

0.679 

103 

224,156 

1999 

Q4 

ACC 

A- 10 

0.722 

107 

235,702 

Independent  Variable:  O&M  Costs 

Our  primary  independent  variable  of  interest  in  this  study  is  O&M  costs. 
Specifically,  we  are  interested  in  those  costs  that  can  be  directly  attributed  to  supporting 
the  flying  operations  of  a  given  MD  or  MDS.  For  this  reason,  the  availability  and 
reliability  of  data  for  this  variable  determine  the  scope  of  our  research. 

Element  of  Expense/Investment  Code  (EEIC)  644,  also  known  as  Material 
Support  Division  (MSD),  contain  costs  directly  associated  with  the  flying  hour  program. 
Costs  for  EEIC  644  can  be  further  disaggregated  by  transaction  type.  Fly-depot  level 
repairables  (DLRs)  are  recorded  as  transaction  type  XD2,  consumable  items  are  coded 
XB3,  and  base  level  repairable  items  are  labeled  XF3.  Transactions  may  occur  as 
charges  or  credits;  however,  our  data  only  consider  the  resulting  net  costs.  Additional 
consumable  costs  are  found  in  EEIC  609  (General  Supply  Division),  but  these  costs  are 
not  considered  because  their  range  extends  far  beyond  flying  operations.  Furthermore, 
EEIC  645  contains  other  DLR  costs,  but  we  exclude  these  costs  from  our  study  as  well 
because  they  are  not  “fly-DLRs.”  It  is  also  worth  noting  that  CLS  costs  are  accounted  for 
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in  EEIC  578,  but  as  we  mentioned  previously,  CLS  aircraft  will  not  be  included  in  our 
research.  Our  study  only  considers  those  costs  contained  in  EEIC  644  because  it 
represents  the  segment  of  O&M  costs  that  are  directly  attributable  to  flying  operations. 

EEIC  644  costs  in  AFTOC  are  simultaneously  allocated  to  the  MDS  they  were 
used  to  support,  indicated  by  the  “Mission  Design  Series — Standard  Reporting 
Designator  (MDS  SRD)”  column  heading,  in  addition  to  the  weapon  system  whose 
program  element  code  (PEC)  was  used  to  pay  the  bill,  reflected  by  the  “Mission 
Design — Cost  Analysis  Improvement  Group  (MD  CAIG)”  column  heading.  In  the 
majority  of  circumstances  these  two  fields  match,  in  which  case  there  is  no  problem. 
When  the  MDS  SRD  field  does  not  match  the  MD  CAIG  field,  we  simply  allocate  the 
dollars  to  the  weapon  system  identified  in  the  MDS  SRD  field  because  that  is  the  system 
supported  by  the  expenditure.  However,  two  cases  arise  that  force  us  to  make  some 
judgmental  allocations  based  on  the  infonnation  available.  When  the  MDS  SRD  field 
contains  “null”  (no  data  entered),  we  look  to  the  MD  CAIG  field  for  information.  If  there 
is  a  weapon  system  identified  in  the  MD  CAIG  field,  we  allocate  the  costs  to  that  weapon 
system  as  the  next  best  alternative  based  on  the  advice  of  contractors  who  maintain  the 
AFTOC  database.  In  the  rare  cases  where  both  fields  are  “null,”  we  discard  the  data 
since  we  have  no  means  of  identifying  the  correct  MDS.  Table  5  shows  an  example  of 
the  raw  cost  data  retrieved  from  AFTOC. 
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Table  5:  Subset  of  Raw  Cost  Data  from  AFTOC 


FY 

QTR 

CAIG 

MAJCOM 

MDS  SRD 

MD  CAIG 

ERRC 

Net  Cost 

1999 

Q3 

2.3.1 

USAFE 

KC-135T 

KC-135R 

XD2 

$13,930.53 

1999 

Q4 

2.3.1 

ACC 

NULL 

NULL 

XB3 

$19,703.22 

1999 

Q4 

2.3.1 

ACC 

NULL 

KC-135R 

XF3 

$1,117.90 

1999 

Q4 

2.3.1 

ACC 

A-10 

A-10 

XB3 

$154,744.15 

1999 

Q4 

2.3.1 

ACC 

A- 10 

A-10 

XD2 

$20,138,457.23 

1999 

Q4 

2.3.1 

ACC 

A-10 

A-10 

XF3 

$855,392.66 

1999 

Q4 

2.3.1 

ACC 

A-10 

F-15E 

XD2 

$3,294.48 

1999 

Q4 

2.3.1 

ACC 

A-10 

F-16C/D 

XB3 

$1,683.69 

1999 

Q4 

2.3.1 

ACC 

A-10 

F-16C/D 

XD2 

$694,477.35 

The  cost  data  for  the  weapon  systems  we  evaluate  include  1 1,745  rows  of  data. 
MDS  SRD  contains  3,593  values  that  are  “null”  (30.6%  of  the  data);  however,  only  174 
entries  contain  “null”  in  both  MDS  SRD  and  MD  CAIG  (1.5%  of  the  data).  This  means 
that  we  relied  on  our  allocation  heuristic  to  account  for  29.1%  of  the  cost  data  and  we 
discarded  an  additional  1.5%  of  the  data. 

As  a  final  means  of  nonnalizing  the  data,  we  used  inflation  rates  approved  by  the 
Office  of  the  Secretary  of  Defense  to  convert  all  costs  to  CY08$. 

Additional  Independent  Variables 

In  order  to  have  adequate  information  to  build  explanatory  models,  we  collected 
data  on  variables  that  have  been  shown  in  previous  research  to  have  predictive  ability 
with  respect  to  AA.  The  following  sections  detail  our  remaining  independent  variables. 

Usage  Variables.  We  obtained  quarterly  flying  hour,  sortie,  and  landing  data  for 
each  MDS,  by  MAJCOM  from  the  REMIS  database.  REMIS  assigns  the  data  both  to  the 
command  that  owns  the  aircraft,  and  to  the  command  that  is  operating  the  aircraft.  As 
one  would  expect,  the  owning  command  and  the  operating  command  are  the  same  in  the 
vast  majority  of  situations.  However,  circumstances  arise  where  one  unit  may  loan 
aircraft  to  another  unit  for  a  given  period.  In  this  situation,  we  ignore  the  operating 
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command  and  allocate  the  usage  data  to  the  command  possessing  the  aircraft.  This 
heuristic  follows  the  same  logic  we  discussed  previously  concerning  the  allocation  of 
O&M  cost  data.  Table  6  shows  an  example  of  the  raw  usage  data  obtained  from  REMIS. 


Table  6:  Subset  of  Raw  Usage  Data  from  REMIS 


FY 

System 

OperatingAgency 

PossCmd 

FHQ4 

SortiesQ4 

LandingsQ4 

FY2007 

F-15D 

PACAF 

PACAF 

392.0 

265 

266 

FY2007 

F-15D 

USAFE 

USAFE 

109.1 

85 

87 

FY2007 

F-16C 

ACC 

ACC 

7,855.7 

5,582 

5,585 

FY2007 

F-16C 

A  ETC 

AETC 

6,027.4 

4,622 

4,631 

FY2007 

F-16C 

AFRC 

ACC 

886.7 

593 

593 

FY2007 

F-16C 

PACAF 

PACAF 

9,999.7 

5,339 

5,357 

FY2007 

F-16C 

USAFE 

USAFE 

6,228.8 

4,181 

4,181 

FY2007 

F-16D 

ACC 

ACC 

957.8 

690 

690 

Inventory  Variables.  LIMS-EV  contains  data  for  average  aircraft  age,  average 
airframe  hours,  TAI,  and  PAI.  As  we  discussed  in  Chapter  II,  TAI  and  PAI  are  two 
different  concepts.  However,  through  conversations  with  LIMS-EV  customer  support 
representatives  and  analysts  in  the  AF/A4LY  (Weapons  System  Division),  we  learned 
that  LIMS-EV  does  not  reflect  these  terms  accurately.  At  the  time  of  this  writing,  data 
provided  from  LIMS-EV  under  the  column  heading  TAI  is  erroneous  and  the  actual  TAI 
is  represented  under  the  “Poss’d”  column  heading.  With  that  said,  we  acquired  quarterly 
data  for  TAI  and  monthly  data  for  average  age  and  average  airframe  hours  for  each  MDS, 
by  MAJCOM.  In  order  to  remain  consistent  with  the  periods  of  data  we  collected  for  the 
other  independent  variables  and  the  dependent  variable,  we  translate  the  monthly  data 
into  quarterly  periods  by  using  only  the  observations  for  the  last  month  of  each  quarter 
(December,  March,  June,  and  September).  Admittedly,  we  could  have  averaged  the  data 
over  each  quarter  to  achieve  the  same  objective,  but  for  ease  of  computation  and  data 
aggregation,  we  took  the  former  approach.  Furthermore,  average  age  exhibits  a  perfect 
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linear  trend  over  time,  which  means  our  analysis  is  not  affected  by  the  choice  we  made  in 
nonnalizing  our  data.  Table  7  shows  an  example  of  the  raw  inventory  data  from  LIMS- 
EV. 


Table  7:  Subset  of  Raw  Inventory  Data  from  LIMS-EV 


Command 

MDS 

Month-Year 

TAI 

Avg  Airframe  Hours 

Avg  Age  of  Fleet 

ACC 

A010A 

OCT-2005 

88 

8583.3 

24.22 

ACC 

A010A 

NOV- 2005 

88 

8622.7 

24.31 

ACC 

A010A 

DEC-2005 

88 

8662.5 

24.4 

ACC 

A010A 

JAN-2006 

88 

8696.3 

24.48 

ACC 

A010A 

FEB-2006 

88 

8725.4 

24.55 

ACC 

A010A 

MAR-2006 

88 

8759.1 

24.63 

ACC 

A010A 

APR-2006 

88 

8789.5 

24.72 

ACC 

A010A 

MAY-2006 

88 

8820.1 

24.81 

Maintenance  Variables.  We  retrieved  maintenance  data  from  LIMS-EV  that 
represent  the  various  states  of  disrepair  aircraft  may  be  coded.  Specifically,  we  collected 
data  on  cannibalization  hours  and  the  five  possible  statuses  of  non-available  aircraft, 
which  are  the  depot,  NMCM,  NMCS,  NMCB,  and  UPNR  rates.  We  do  not  include  the 
variables  representing  non-availability  in  our  models  since  they  would  not  provide  any 
useful  explanatory  information,  but  we  do  use  cannibalization  hours  as  an  independent 
variable.  Although  cannibalization  hours  represent  non-availability,  the  practice  of 
cannibalizing  aircraft  is  a  proven  method  of  providing  necessary  parts  to  other  aircraft  in 
the  fleet  (Oliver,  2001). 

Personnel  Variables.  AFTOC  maintains  annual  data  on  the  actual  number  of 
personnel  assigned  to  support  a  given  MDS  by  AFSC  for  both  officers  and  enlisted.  We 
obtained  data  reflecting  specifically  maintenance  personnel  for  our  study.  We  further 
disaggregated  the  data  by  separating  the  data  for  officers  from  enlisted  and  also  grouping 
the  enlisted  data  by  skill  level  (1,  3,  5,  7,  9,  or  0).  Enlisted  personnel  typically  enter  a 
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career  field  as  a  skill  level  one  and  progress  upwards  based  on  their  time  served  in  that 
field  and  their  demonstrated  level  of  expertise;  chief  master  sergeants  are  automatically 
designated  as  skill  level  zero.  Disaggregating  the  data  in  this  manner  allows  us  to 
analyze  more  relationships  such  as  the  effect  that  higher  proportions  of  low  skill  level 
maintainers  have  on  AA  rates.  In  order  to  translate  the  annual  periods  into  quarterly 
periods,  we  reason  that  the  number  of  assigned  personnel  fluctuates  little  over  the  course 
of  a  year  so  we  simply  use  the  annual  figures  for  all  four  quarters  of  the  year.  Table  8 
shows  an  example  of  the  raw  personnel  data  retrieved  from  AFTOC. 


Table  8:  Subset  of  Raw  Personnel  Data  from  AFTOC 


FY 

CAIG 

COMMAND 

MDS 

AFSC 

AFSC COUNT 

FY1998 

1.2 

ACC 

A-10A 

2W151 

109.4859 

FY1998 

1.2 

ACC 

A-10A 

2W171 

43.1567 

FY1998 

1.2 

ACC 

A-10A 

2W191 

7.3694 

FY1998 

1.2 

ACC 

B-1B 

2AOOO 

1 

FY1998 

1.2 

ACC 

B-1B 

2AOOO 

2 

FY1998 

1.2 

ACC 

B-1B 

2A031 

54.75 

FY1998 

1.2 

ACC 

B-1B 

2A051 

0.75 

FY1998 

1.2 

ACC 

B-1B 

2A051 

58 

Dummy  Variables:  Location,  Season,  and  Aircraft  Characteristics.  We  created 
dummy  variables  to  represent  the  location,  season,  and  characteristics  of  our  MDS.  We 
use  four  location  dummy  variables  to  represent  the  five  MAJCOMs  and  three  seasonal 
dummy  variables  to  represent  the  four  quarters  in  our  study.  We  also  create  dummy 
variables  to  represent  each  of  the  MDS  groups  and  aircraft  types  in  our  study. 

Table  9  provides  a  list  of  the  dummy  variables  we  created. 
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Table  9:  List  of  Dummy  Variables 


MAJCOM 

Quarter 

MDS  Group 

Aircraft  Type 

AETC 

AMC 
PACAF 
USAFE 
(ACC  is  the  base 
case) 

Q2 

Q3 

Q4 

(Quarter  1  is  the  base 
case) 

B-1B 

B-2A 

B-52H 

F-15C/D 

F-15E 

F-16C/D 

KC-135 

T-38 

(The  A- 10  group  is 
the  base  case) 

Bomber 

Fighter/Attack 

Heavy 

(Trainers  are  the  base 
case) 

Data  Aggregation 

Our  data  can  be  analyzed  on  three  separate  dimensions:  1)  type  of  aircraft,  2) 
level  of  assignment  of  aircraft,  and  3)  units  of  time.  First,  it  is  possible  to  aggregate 
aircraft  data  at  a  high  level  based  on  aircraft  type  (e.g.  fighter  or  bomber)  or  at  lower 
levels  such  as  MD  (e.g.  F-15  or  A- 10)  or  MDS  (e.g.  F-16C  or  F-15E).  In  his  dissertation 
published  by  RAND  in  2008,  Lt  Col  Eric  Unger  argued  that  O&M  costs  at  the  MDS  level 
in  AFTOC  suffer  from  data  validity  concerns.  Specifically,  he  found  that  some  costs 
were  allocated  from  the  MD  level  to  the  MDS  level  based  on  proportion  of  flying  hours 
instead  of  actual  expenditures  for  each  MDS  within  the  given  MD.  Consequently,  some 
data  may  be  misallocated  within  the  MD  and  using  the  data  in  explanatory  models  would 
result  in  an  overstatement  of  the  relationship  between  flying  hours  and  costs  (Unger, 
2008).  Therefore,  we  evaluate  aircraft  at  the  MDS  level  only  where  AFTOC  properly 
allocates  costs. 

In  his  2008  AFIT  thesis,  lLt  Tyler  Hess  created  cost  forecasting  models  for  the 
Air  Force  flying  hour  program.  His  research  built  on  Unger’s  findings  such  that  he  was 
able  to  analyze  aircraft  at  the  lowest  possible  level  while  still  maintaining  proper  cost 
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allocation.  We  build  on  Hess’s  research  and  use  the  same  MDS  groupings  that  he  proved 
properly  represented  O&M  costs  (Hess,  2009).  Table  10  shows  the  MDS  in  our  study 
and  their  final  MDS  grouping  for  our  research. 


Table  10:  Assignment  of  MDS  to  MDS  Groups 


MDS 

MDS  Grouping 

A-10A,  OA-10A 

A-10 

B-1B 

B-1B 

B-2A 

B-2A 

B-52H 

B-52H 

F-15C,  F-15D 

F-15C/D 

F-15E 

F-15E 

F-16C,  F-16D 

F-16C/D 

KC-135R,  KC-135T 

KC-135 

T-38A,  T-38C 

T-38 

Second,  our  data  can  be  analyzed  at  different  levels  of  aggregation  based  on 
location.  For  example,  data  can  be  acquired  as  low  as  the  base  level,  it  can  be  aggregated 
to  the  MAJCOM  level,  or  it  can  be  further  aggregated  and  analyzed  at  the  Air  Force 
level.  Hess  found  that  analyzing  cost  and  usage  data  at  the  base  level  presents  construct 
validity  concerns.  The  crux  of  his  argument  was  that  costs  are  often  misallocated  at  the 
base  level  of  aggregation  because  organizations  often  pay  for  things  that  go  towards 
supporting  aircraft  that  they  do  not  own.  By  moving  from  the  base  to  the  MAJCOM 
level  of  aggregation,  we  are  able  to  avoid  much  of  the  misallocation  (Hess,  2009).  Thus, 
we  choose  to  analyze  our  data  at  the  MAJCOM  level  of  aggregation  for  location. 

Time  is  the  final  dimension  for  which  our  data  can  be  aggregated.  Typically,  data 
are  available  in  monthly,  quarterly,  or  annual  periods.  In  order  to  have  sufficient  data 
points  for  our  analysis,  we  collected  our  data  at  the  quarterly  level  of  aggregation. 
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Final  Database 


Once  our  data  was  normalized  and  aggregated  at  the  quarterly  level,  we  used 
Microsoft  Excel1'  and  Microsoft  Access®  to  create  our  final  database.  Table  1 1  shows 
selected  variables  from  the  final  database. 


Table  11:  Subset  of  Variables  from  Final  Database 


FY 

Q 

MAJCOM 

MDS 

AA  Rate 

Total  EEIC  644 

Fly  Hrs 

Sorties 

Landings 

TAI 

Avg  Age 

CANN  Hrs 

1998 

Q1 

ACC 

A- 10 

0.725 

14,727,747 

15,556 

7,978 

7,978 

162 

16 

3,617 

1998 

Q2 

ACC 

A- 10 

0.671 

18,755,389 

15,845 

8,352 

8,372 

162 

17 

3,643 

1998 

Q3 

ACC 

A- 10 

0.692 

20,649,961 

17,120 

8,674 

8,687 

161 

17 

4,642 

1998 

Q4 

ACC 

A- 10 

0.703 

26,734,875 

16,147 

7,942 

7,944 

153 

17 

3,933 

1999 

Q1 

ACC 

A-10 

0.705 

12,622,025 

13,367 

7,095 

7,096 

152 

17 

3,909 

1999 

Q2 

ACC 

A- 10 

0.668 

18,801,395 

14,280 

7,646 

7,646 

152 

18 

3,750 

1999 

Q3 

ACC 

A-10 

0.679 

22,469,193 

14,346 

7,211 

7,250 

151 

18 

4,668 

1999 

Q4 

ACC 

A-10 

0.722 

26,007,053 

13,642 

7,210 

7,311 

144 

18 

4,534 

Variable  Analysis  Methodology 

Because  of  the  large  number  of  independent  variables  we  obtained  data  for,  we 
must  investigate  the  potential  predictive  ability  between  each  of  them  and  the  dependent 
variable  before  we  attempt  to  build  models.  Furthermore,  we  are  interested  in  accounting 
for  the  possibility  that  lagging  relationships  may  exist  between  the  independent  variables 
and  the  dependent  variables.  To  test  for  this  condition,  we  lag  each  independent  variable 
with  respect  to  time  one  to  four  quarters  into  the  future.  These  variables  will  depict  the 
relationship  between  an  independent  variable  in  one  quarter  and  the  dependent  variable  in 
future  quarters. 

Correlation  analysis  is  a  convenient  technique  for  expediently  examining  the 
linear  relationships  between  variables.  Using  JMP®  release  seven,  we  are  able  to 
construct  a  multivariate  matrix  that  shows  the  linear  relationship  between  every 
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combination  of  two  variables.  We  use  the  correlation  coefficient,  which  is  a  measure  of 
linear  dependency  between  two  variables  on  a  negative  one  to  one  scale,  to  detennine  the 
degree  of  correlation  between  any  two  of  the  variables.  Coefficients  with  an  absolute 
value  of  one  prove  a  perfect  linear  relationship  between  two  variables  (Wooldridge, 

2006). 

In  addition  to  providing  insight  into  which  variables  will  be  useful  in  explanatory 
models,  correlation  analysis  will  help  us  identify  potential  cases  of  multicollinearity. 
Multicollinearity  refers  to  a  correlation  between  independent  variables  in  a  multiple 
regression  model  (Wooldridge,  2006).  Because  instances  of  multicollinearity  add 
confusion  to  a  model  by  making  it  difficult  to  interpret  the  contribution  of  the 
independent  variables,  we  avoid  it  by  not  including  pairs  of  variables  in  our  models 
where  multicollinearity  exists.  In  order  to  identify  specific  cases  of  multicollinearity,  we 
use  JMP®  to  calculate  variance  inflation  factors  (VIF)  for  the  independent  variables  in 
our  models.  In  practice,  acceptable  VIF  levels  are  generally  less  than  or  equal  to  five  or 
ten.  When  VIF  levels  exceed  these  thresholds,  it  is  a  sign  that  a  high  degree  of 
multicollinearity  exists  for  two  or  more  of  the  independent  variables.  For  our  analysis, 
we  accept  VIF  measures  of  less  than  or  equal  to  five. 

We  also  examine  the  distribution  of  the  data  for  each  of  our  variables  to  detennine 
if  there  is  a  need  to  perfonn  discrete  analysis.  Random  variables  that  show  inegular 
patterns  in  their  data  may  provide  better  predictive  ability  if  the  data  is  categorized  into 
discrete  values  (Wooldridge,  2006).  In  such  cases,  we  create  dummy  variables  to 
represent  the  discrete  data. 
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Model  Building  Methodology 

Since  we  have  numerous  variables  that  may  prove  to  be  explanatory  in  predicting 
AA,  we  use  multiple  regression  analysis  to  create  our  models.  Briefly  stated,  multiple 
regression  allows  us  to  simultaneously  control  for  many  variables  when  explaining  the 
response.  In  our  case,  this  is  an  important  attribute  because  we  will  be  able  to  investigate 
the  affect  O&M  costs  have  on  AA,  while  controlling  for  several  other  factors  at  the  same 
time. 

Ordinary  Least  Squares  (OLS)  is  the  most  commonly  used  method  for  estimating 
the  parameters  of  the  regression  model  and  it  is  the  method  we  use  in  our  study.  OLS 
estimates  the  parameters  by  minimizing  the  sum  of  squared  errors  between  the  actual  and 
predicted  values  of  the  model  (referred  to  as  the  residuals).  OLS  provides  the  best  linear 
unbiased  estimator  for  the  parameters,  given  the  assumptions  of  this  technique  are  met. 
We  discuss  the  assumptions  later  in  this  chapter. 

In  a  general  fonn,  multiple  regression  equations  take  the  following  structure: 

Y  =  p0  +  P1X1  +  P2X2  +  . . .  +  PkXk  +  £  (4) 

Where: 

Y  =  dependent  variable 

xi,  x2,  ...  xk  =  independent  variables 

Po  =  the  intercept 

Pi,  P2,  ...  Pk  =  the  population  coefficients 
s  =  the  random  error  component 

Our  models  are  constructed  in  this  fashion  such  that  AA  is  explained  to  the  maximum 
extent  possible.  To  guide  our  analysis,  we  use  the  five  steps  outlined  below  for  building 
valid,  useful  models  (McClave  et  ah,  2008:666): 
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Step  1 :  Hypothesize  the  deterministic  component  of  the  model.  This  component 
relates  the  mean,  E(y),  to  the  independent  variables  xi,  x2,  ...  xk.  This  involves 
the  choice  of  the  independent  variables  to  be  included  in  the  model. 

Step2:  Use  the  sample  data  to  estimate  the  unknown  model  parameters  (3o,  Pi,  P2, 
. . .  Pk  in  the  model. 

Step  3:  Specify  the  probability  distribution  of  the  random  error  term,  s,  and 
estimate  the  standard  deviation  of  this  distribution,  a. 

Step  4:  Check  that  the  assumptions  on  s  are  satisfied,  and  make  model 
modifications  if  necessary. 

Step  5:  Statistically  evaluate  the  usefulness  of  the  model. 


Testing  Regression  Assumptions 

Step  four  in  the  model  building  process  outlined  previously  requires  that  the 
assumptions  of  the  random  error  term,  s,  are  satisfied.  The  assumptions  refer  to  the 
probability  distribution  of  s  and  are  given  as  follows  (McClave  et  ah,  2008:667): 

Assumption  1:  Mean  equal  to  zero. 

Assumption  2:  Variance  equal  to  cT  (also  known  as  constant  variance  or 

homoscedasticity) . 

Assumption  3:  Normal  distribution. 

Assumption  4:  Random  errors  are  independent  (in  a  probabilistic  sense). 

The  validity  of  our  models  relies,  in  part,  on  the  random  error  term  meeting  the 
assumptions  outlined  above.  We  provide  verification  of  the  assumptions  for  all  of  our 
models  and  discuss  deviations  from  the  assumptions  in  Chapter  IV. 

In  order  to  check  that  the  first  assumption  is  met,  we  rely  on  visual  inspection  of  a 
plot  of  the  residuals  about  a  mean  line  of  zero.  We  test  for  compliance  with  the  second 
assumption  by  visually  analyzing  the  residual  by  predicted  plots  to  determine  whether 
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any  irregular  patterns  of  variance  are  present.  In  addition,  we  use  the  Breusch-Pagan  test 
to  statistically  determine  whether  homoscedasticity  exists.  For  this  test,  small  p-values 
reject  the  null  hypothesis  of  constant  variance;  therefore,  large  p-values  are  desired.  We 
test  the  third  assumption  by  plotting  the  studentized  residuals  in  a  histogram  for  visual 
examination,  followed  by  statistical  validation  using  the  Shapiro-Wilk  test.  Similar  to  the 
Breusch-Pagan  test,  large  p-values  are  preferred  so  that  we  may  accept  the  null 
hypothesis  of  nonnality.  Failure  to  meet  the  fourth  assumption  is  caused  by 
autocorrelation  of  the  residuals,  meaning  each  residual  is  affected  by  the  previous  one. 

We  check  for  autocorrelation  by  analyzing  a  plot  of  the  residuals  by  row  in  order  to  see  if 
any  trends  are  obvious.  We  further  test  this  assumption  using  the  Durbin- Watson  test 
which  tests  for  autocorrelation  at  lag  one.  Empirical  evidence  shows  that  data  for  AA 
rates  may  be  subject  to  positive  autocorrelation  (high  residuals  tend  to  be  followed  by 
high  residuals,  and  negative  residuals  tend  to  be  followed  by  negative  residuals)  (Oliver, 
2001).  Given  this  knowledge,  we  use  a  left-tailed  test  for  positive  autocorrelation  and 
reject  the  null  hypothesis  of  independence  whenever  p-values  are  less  than  0.05. 

In  addition  to  the  assumptions  already  discussed,  we  analyze  every  data  point  in 
our  models  using  the  Cook’s  D  Influence  statistic.  This  statistic  measures  the  influence  a 
given  data  point  has  on  the  overall  model.  For  our  analysis,  we  specify  large  Cook’s  D 
measures  as  those  values  over  0.5.  In  cases  where  a  data  point  has  a  Cook’s  D  value 
greater  than  0.5,  we  re-run  the  model  with  that  data  point  excluded  in  order  to  detennine 
whether  it  should  remain  in  the  model.  If  there  are  no  significant  changes  to  the  p-values 
of  the  overall  model  or  individual  parameter  p-values,  we  allow  the  data  point  to  remain 
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in  the  model.  Additionally,  we  attempt  to  provide  an  explanation  for  why  the  data  point 
was  influential. 

Model  Validation 

In  order  to  determine  the  robustness  of  the  predictive  ability  of  our  explanatory 
models,  we  must  validate  our  models.  To  do  this,  we  set  aside  the  final  9  quarters  of  data 
from  our  original  dataset  (20  percent)  while  building  our  models.  Once  our  models  are 
complete,  we  combine  the  independent  variable  data  from  the  final  nine  quarters  with  the 
data  used  to  build  the  model.  The  dependent  variable  data  for  the  last  nine  quarters 
remains  excluded  so  that  when  the  model  is  run  in  JMP®,  we  are  able  to  generate 
prediction  intervals  for  each  of  those  quarters.  Finally,  we  are  able  to  detennine  if  the 
actual  values  of  the  dependent  variable  for  the  nine  quarters  fall  within  the  prediction 
interval  in  addition  to  comparing  the  actual  values  to  the  values  predicted  by  the  model. 
Using  this  procedure  for  model  validation  allows  us  to  evaluate  our  model’s  usefulness 
when  new  data  from  outside  the  original  sample  is  used  for  prediction. 

Chapter  Summary 

In  Chapter  III,  we  outlined  the  scope  of  our  research  effort  and  detailed  the  data 
we  use  to  perform  our  analysis.  We  explained  the  statistical  techniques  we  use  to 
investigate  the  predictive  ability  of  our  data  and  construct  our  explanatory  models. 

Lastly,  we  described  how  we  test  that  the  assumptions  of  our  regression  models  are 
satisfied  and  how  we  validate  the  models’  usefulness.  In  the  next  chapter,  we  describe 
our  analysis  and  detail  the  results. 
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IV:  Analysis  and  Results 


Using  the  methods  described  in  the  previous  chapter,  we  discuss  our  analysis  and 
the  results  of  our  study  in  Chapter  IV.  First,  we  outline  the  challenges  presented  by  our 
data  and  the  techniques  we  use  to  overcome  the  problems.  Next,  we  evaluate  the  models 
created  for  each  of  the  MDS  in  our  study.  Finally,  we  summarize  our  results  and  discuss 
other  techniques  that  we  explore  for  establishing  a  relationship  between  O&M  costs  and 
AA. 

Adjustments  to  Data  Required  for  Analysis 

We  begin  by  estimating  models  for  each  MDS  by  MAJCOM  with  AA  hours  as 
the  dependent  variable  and  total  EEIC  644  costs,  flying  hours,  sorties,  landings,  TAI,  and 
average  age  as  the  independent  variables.  Not  surprisingly,  we  encounter  problems  with 
multicollinearity  for  several  of  the  independent  variables.  Table  12  is  a  correlation 
matrix  of  the  variables  in  the  model  for  ACC  F-15C/Ds.  Although  the  data  is  different 
for  every  MDS  and  MAJCOM,  the  data  we  show  for  ACC  F-15C/Ds  is  representative  of 
the  correlations  we  find  with  nearly  all  of  the  other  MDS  in  our  study.  Thus,  we  use  this 
regression  output  as  an  example  to  explain  the  common  problems  encountered  with  all  of 
the  MDS  in  our  study. 
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Table  12:  Correlation  Matrix  of  Variables  for  ACC  F-15C/Ds 


AAHrs 

Total 
EEIC  644 

Fly  Hrs 

Sorties 

Landings 

TAI 

Avg  Age 

AAHrs 

1.0000 

0.2620 

0.8293 

0.8727 

0.8722 

0.8940 

-0.5135 

Total 

EEIC  644 

1.0000 

0.1047 

0.2348 

0.2344 

0.1924 

0.1589 

Fly  Hrs 

1.0000 

0.8633 

0.8637 

0.8808 

-0.7820 

Sorties 

1.0000 

1.0000 

0.8755 

-0.6050 

Landings 

1.0000 

0.8752 

-0.6057 

TAI 

1.0000 

-0.7489 

Avg  Age 

1.0000 

Due  to  their  obvious  operational  relationships,  flying  hours,  sorties,  and  landings 
exhibit  very  high  degrees  of  correlation.  In  fact,  sorties  and  landings  are  perfectly 
correlated  for  this  dataset  as  exhibited  with  a  correlation  coefficient  of  1 .00,  while  flying 
hours  yields  correlation  coefficients  of  0.8633  and  0.8637  with  sorties  and  landings, 
respectively  (highlighted  in  bold  in  Table  12).  Additionally,  we  calculate  VIF  scores  for 
these  three  variables  as  shown  in  Table  13;  all  measures  exceed  our  threshold  of  five. 


Table  13:  VIF  Scores  for  Usage  Variables  in  Initial  ACC  F-15C/D  Model 


Term 

VIF 

Fly  Hrs 

7.455 

66724.138 

Landings 

66644.096 

Therefore,  we  detennine  that  it  would  be  unwise  to  include  more  than  one  of  the 
usage  variables  in  our  models  since  doing  so  would  make  it  difficult  to  interpret  the 
contribution  of  each  variable  to  the  model.  As  a  rule,  we  elect  to  use  flying  hours  as  our 
usage  variable  of  choice  in  all  models  where  it  demonstrates  predictive  ability. 

Next,  we  see  that  changes  in  TAI  over  time  in  our  dataset  result  in  overstated 
relationships  between  AA  hours  and  nearly  all  of  the  independent  variables  in  our  dataset, 
except  average  age.  The  reason  for  this  is  simple.  TAI  fluctuates  as  the  result  of  several 
factors  such  as  organizational  or  mission  changes  within  operational  Air  Force  units  and 
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the  retirement  or  acquisition  of  new  aircraft.  When  these  events  occur,  it  is  natural  for 
other  changes  to  take  place  such  as  movement  of  personnel  and  resources.  If  the  number 
of  aircraft  is  reduced,  it  follows  that  the  number  of  operators  and  maintainers  will 
decrease  as  well,  resulting  in  fewer  flying  hours  and  fewer  total  available  hours.  In  order 
to  address  this  problem,  we  convert  all  of  our  variables,  with  the  exception  of  average 
age,  into  a  rate  of  some  kind.  AA  is  commonly  expressed  as  a  percentage  so  it  makes 
sense  to  use  the  AA  rate  as  our  dependent  variable.  For  all  of  our  independent  variables 
(except  age),  we  divide  each  data  point  by  TAI  which  results  in  variables  represented  as  a 
“per-aircraff  ’  rate.  This  procedure  results  in  variables  that  show  genuine  relationships  in 
our  models  and  allow  for  direct  comparison  across  MAJCOMs  and  MDS. 

Explanatory  Models 

Using  the  model  building  process  outlined  in  the  previous  chapter,  we  developed 
models  for  16  of  our  22  MDS  by  MAJCOM  by  quarter  datasets.  Of  those  16  models, 
only  2  found  EEIC  644  costs  as  a  predictive  variable.  Nonetheless,  we  discuss  the  four 
best  models  in  detail  (those  with  an  adjusted  R2  of  greater  than  0.70),  and  then  summarize 
the  remaining  models.  Appendix  A  contains  a  complete  review  of  our  results. 

KC-135  by  AMC  by  Quarter 

Table  14  highlights  the  summary  statistics  of  our  model  for  KC-135  aircraft  in  Air 
Mobility  Command  (AMC).  Only  flying  hours  per  TAI  and  the  dependent  variable 
lagged  one  period  are  included  as  independent  variables.  We  utilize  the  dependent 
variable  lagged  one  period  in  this  model  because  without  it  we  notice  first  order 
autocorrelation  with  a  Durban-Watson  statistic  of  0.46,  which  is  significant  at  the  0.01 
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level.  Additionally,  the  model’s  adjusted  R2  falls  from  0.86945  to  0.5 1178  without  the 
lagged  dependent  variable  included  in  the  model. 

Prior  to  accepting  this  specification  as  a  useful  model,  we  verified  that  the 
assumptions  of  nonnality,  constant  variance,  and  independence  were  met  as  described  in 
the  previous  chapter.  Additionally,  we  checked  for  influential  data  points  using  the 
Cook’s  D  test  and  found  that  no  observations  were  influential.  We  provide  the  results  of 
our  diagnostic  tests  for  this  model  in  Appendix  B. 


Table  14:  KC-135  (AMC)  Explanatory  Model 


Summary  of  Fit 

R2 

0.87736 

Adjusted  R2 

0.86945 

Root  Mean  Square  Error 

0.02655 

Mean  of  Response 

0.60528 

Observations  (or  Sum  Wgts) 

34 

Analysis  of  Variance 

Source 

DF 

Sum  of  Squares  Mean  Square 

F  Ratio 

Model 

2 

0.15627 

0.07813 

110.887 

Error 

31 

0.02184 

0.00070  Prob  >  F 

C.  Total 

33 

0.17812 

<.0001 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

VIF 

Intercept 

0.12256 

0.04113 

2.98 

0.0056 

FlyHrs/TAI 

0.00061 

0.00015 

3.99 

0.0004 

1.807 

lQLagAARate 

0.67384 

0.08413 

8.01 

<.0001 

1.807 

To  test  the  robustness  and  validity  of  the  model,  we  include  the  final  nine  periods 
of  independent  variable  data  and  run  the  model  in  JMP  “ .  This  process  generates 
individual  prediction  intervals  for  the  dependent  variable  at  each  of  the  nine  periods. 
Theoretically,  our  model  should  be  able  to  predict  AA  rates  within  the  prediction 
intervals  (at  95%  confidence)  95%  of  the  time.  We  show  the  results  of  this  analysis  in 
Table  15  and  Figure  7. 
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Table  15:  KC-135  (AMC)  Sensitivity  Analysis 


FY-Q 

Lower  95% 
Confidence 

Observed  AA 
Rate 

Predicted  AA 
Rate 

Upper  95% 
Confidence 

Absolute 
Percent  Error 

2006-Q4 

0.6524 

0.6875 

0.7095 

0.7666 

3.19% 

2007-Q1 

0.6294 

0.6235* 

0.6854 

0.7414 

9.91% 

2007-Q2 

0.5908 

0.6223 

0.6474 

0.7039 

4.02% 

2007-Q3 

0.6029 

0.6436 

0.6618 

0.7208 

2.83% 

2007-Q4 

0.6285 

0.6351 

0.6892 

0.7500 

8.53% 

2008-Q1 

0.6104 

0.6337 

0.6686 

0.7268 

5.50% 

2008-Q2 

0.6168 

0.6112* 

0.6767 

0.7366 

10.72% 

2008-Q3 

0.6035 

0.6619 

0.6652 

0.7268 

0.49% 

2008-Q4 

0.6487 

0.6454* 

0.7108 

0.7729 

10.14% 

*Observations  fall  outside  prediction  interval 

MAPE  =  6.15% 

Although  this  was  our  best  model  with  respect  to  the  adjusted  R"  value,  empirical 
results  show  that  the  observed  AA  rates  fell  within  the  prediction  interval  only  66.7%  of 
the  time.  We  should  also  note  that  the  prediction  intervals  had  an  average  range  of  0.12, 
which  means  that  our  model  should  predict  AA  rates  within  a  window  of  12  percentage 
points.  Given  that  this  could  mean  the  difference  between  achieving  a  stated  goal  for  AA 
or  falling  well  short,  the  prediction  interval  range  produced  by  our  model  may  be  too 
large  to  be  useful  for  Air  Force  decision  makers.  Additionally,  the  mean  absolute  percent 
error  (MAPE)  for  this  model  was  6. 15%. 
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Figure  7:  KC-135  (AMC)  Sensitivity  Analysis 

Figure  7  is  useful  because  it  visually  shows  the  trend  of  AMC  KC-135  AA  rates 
over  the  last  four  years.  We  can  see  that  the  AA  rates  observed  during  the  last  nine 
quarters  are  mostly  lower  than  previous  quarters.  Additionally,  our  model  failed  to 
predict  this  trend.  One  possible  reason  is  that  the  range  of  independent  variable  data  used 
to  produce  the  model  does  not  reflect  the  range  of  data  used  to  validate  the  model.  As  a 
rule,  a  model's  usefulness  will  suffer  if  it  is  used  to  predict  a  response  outside  the  range  of 
data  from  which  it  was  created.  Specifically,  the  data  we  used  when  constructing  the 
model  for  the  flying  hours  per  TAI  variable  ranged  from  a  low  of  67.6  to  a  high  of  184.9. 
However,  the  data  used  for  this  variable  when  we  validated  the  model  exceeded  this 
threshold  seven  times  with  values  ranging  from  187.9  to  232.4.  We  show  the  ranges  of 
independent  variable  data  in  Appendix  C. 

F-15E  by  ACC  by  Quarter 

Our  next  model  is  for  F-15Es  in  Air  Combat  Command  (ACC).  Table  16  shows 
the  summary  statistics,  which  include  an  adjusted  R“  of  0.80918.  We  use  cannibalization 
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hours  per  TAI,  the  total  number  of  1-,  3-,  and  5-level  maintainers  per  TAI,  and  the 
dependent  variable  lagged  one  period  as  independent  variables.  Again,  we  include  the 
lagged  dependent  variable  in  this  model  because  without  it  we  notice  first  order 
autocorrelation  with  a  Durban-Watson  statistic  of  1.46,  which  is  significant  at  the  0.05 
level.  Moreover,  the  model’s  adjusted  R  drops  to  0.75472  without  the  lagged  dependent 
variable  included  in  the  model. 

Just  as  we  did  with  the  first  model,  we  verified  that  the  assumptions  of  normality, 
constant  variance,  and  independence  were  met,  in  addition  to  checking  for  influential  data 
points.  Yet  again,  all  assumptions  were  satisfied. 


Table  16:  F-15E  (ACC)  Explanatory  Model 


Summary  of  Fit 

R2 

0.82653 

Adjusted  R2 

0.80918 

Root  Mean  Square  Error 

0.01708 

Mean  of  Response 

0.67059 

Observations  (or  Sum  Wgts) 

34 

Analysis  of  Variance 

Source 

DF 

Sum  of  Squares  Mean  Square 

F  Ratio 

Model 

3 

0.04173 

0.01391 

47.647 

Error 

30 

0.00875 

0.00029  Prob  >  F 

C.  Total 

33 

0.05049 

<.0001 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

VIF 

Intercept 

0.31488 

0.08637 

3.65 

0.0010 

lQLagAARate 

0.38525 

0.12231 

3.15 

0.0037 

2.555 

CANNhrs/TAl 

-0.00105 

0.00022 

-4.73 

<.0001 

2.228 

1,3,5  SkillLvl/TAI 

0.01015 

0.00454 

2.23 

0.0333 

1.274 

Using  the  same  process  as  before,  we  test  the  validity  of  our  model  by  including 
the  final  nine  periods  of  independent  variable  data  and  running  the  model  in  JMP®.  Table 
17  and  Figure  8  provide  the  results  of  this  analysis. 
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Table  17:  F-15E  (ACC)  Sensitivity  Analysis 


FY-Q 

Lower  95% 
Confidence 

Observed  AA 
Rate 

Predicted  AA 
Rate 

Upper  95% 
Confidence 

Absolute 
Percent  Error 

2006-Q4 

0.6689 

0.6740 

0.7050 

0.7412 

4.61% 

2007-Q1 

0.6735 

0.6788 

0.7141 

0.7547 

5.20% 

2007-Q2 

0.6742 

0.7108 

0.7128 

0.7514 

0.28% 

2007-Q3 

0.6870 

0.6899 

0.7248 

0.7625 

5.05% 

2007-Q4 

0.6792 

0.6632* 

0.7184 

0.7575 

8.32% 

2008-Q1 

0.6679 

0.5930* 

0.7085 

0.7490 

19.47% 

2008-Q2 

0.6310 

0.6737 

0.6801 

0.7292 

0.95% 

2008-Q3 

0.6741 

0.6918 

0.7141 

0.7541 

3.22% 

2008-Q4 

0.6797 

0.6324* 

0.7181 

0.7564 

13.55% 

*Observations  fall  outside  prediction  interval 

MAPE  =  6.74% 

Our  model  for  F-15Es  in  ACC  perfonns  nearly  as  well  as  our  first  model.  Again, 
six  out  of  nine  (66.7%)  observations  fall  within  the  prediction  interval;  however,  the 
MAPE  is  slightly  larger  at  6.74%.  Furthermore,  the  range  of  our  prediction  intervals  is 
smaller  than  the  first  model  with  an  average  of  0.08  (i.e.,  8%  in  terms  of  the  AA  rates). 
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Figure  8:  F-15E  (ACC)  Sensitivity  Analysis 

Figure  8  shows  a  sharp  decrease  in  the  AA  rate  during  the  first  quarter  of  2008 
that  our  model  did  not  predict.  In  our  model’s  defense,  this  observation  may  be 
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considered  somewhat  of  an  outlier.  After  mechanical  failure  resulted  in  the  crash  of  an 


Air  National  Guard  F-15C  on  November  2,  2007,  the  Air  Force  grounded  all  F-15  models 
until  an  investigation  was  conducted.  Later,  on  November  15,  2007,  the  Air  Force  began 
lifting  the  restriction  for  F-15Es  after  they  concluded  that  the  E-model  was  not 
susceptible  to  the  same  failure.  Nonetheless,  F-15Es  were  still  grounded  for  at  least  13 
days  which  resulted  in  a  low  AA  rate  for  the  first  quarter  of  2008  (Wicke,  2007).  Finally, 
we  must  consider  that  the  independent  variable  data  used  to  validate  the  model  fell 
outside  the  range  of  data  used  to  construct  the  model  six  times  for  the  cannibalization 
hours  per  TAI  variable  and  once  for  the  total  number  of  1-,  3-,  and  5 -level  maintainers 
per  TAI  variable. 

B-l  by  ACC  by  Quarter 

Our  third  model  is  for  B-l  aircraft  in  ACC.  Table  18  provides  the  summary 
statistics  for  this  model.  We  use  the  three  regressors  from  the  first  two  models,  in 
addition  to  the  dependent  variable  lagged  one  period  as  independent  variables  for  our  B-l 
model.  Due  to  first  order  autocorrelation  as  evidenced  by  a  Durban- Watson  statistic  of 
0.75  (significant  at  the  0.01  level),  we  included  the  lagged  dependent  variable. 
Additionally,  the  model’s  adjusted  R2  decreases  from  0.76778  to  0.61946  without  the 
lagged  dependent  variable  included  in  the  model.  Interestingly,  the  coefficient  for  the 
total  number  of  1-,  3-,  and  5-level  maintainers  per  TAI  is  negative  in  this  equation  where 
it  was  positive  in  the  model  for  F-15Es.  We  suspect  that  as  the  total  number  of  lower 
skilled  maintainers  reaches  a  tipping  point,  they  begin  to  adversely  affect  AA  rates, 
especially  if  the  total  number  of  highly  skilled  maintainers  does  not  increase  at  the  same 
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rate.  In  this  situation,  the  total  number  of  maintainers  may  be  constant,  but  the  proportion 
of  low-skill  maintainers  will  be  increasing. 

As  we  did  with  the  first  two  models,  we  verified  that  the  assumptions  of 
nonnality,  constant  variance,  and  independence  were  met,  in  addition  to  checking  for 
influential  data  points.  Here  again,  all  assumptions  were  fulfilled. 


Table  18:  B-l  (ACC)  Explanatory  Model 


Summary  of  Fit 

R2 

0.79593 

Adjusted  R2 

0.76778 

Root  Mean  Square  Error 

0.03423 

Mean  of  Response 

0.50145 

Observations  (or  Sum  Wgts) 

34 

Analysis  of  Variance 

Source 

DF 

Sum  of  Squares  Mean  Square 

F  Ratio 

Model 

4 

0.13255 

0.03313 

28.278 

Error 

29 

0.03398 

0.00117  Prob  >  F 

C.  Total 

33 

0.16654 

<.0001 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

VIF 

Intercept 

0.54769 

0.14720 

3.72 

0.0008 

FlyHrs/TAI 

0.00177 

0.00067 

2.61 

0.0141 

3.672 

CANNhrs/TAl 

-0.00139 

0.00043 

-3.23 

0.0031 

3.705 

l,3,5_SkillLvl/TAI 

-0.01053 

0.00378 

-2.78 

0.0094 

5.148 

lQLagAARate 

0.61311 

0.13618 

4.50 

0.0001 

2.579 

Once  again,  we  test  the  validity  of  our  model  by  including  the  final  nine  periods 
of  independent  variable  data  and  running  the  model  in  JMP®.  Table  19  and  Figure  9 
provide  the  results  of  this  analysis. 
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Table  19:  B-l  (ACC)  Sensitivity  Analysis 


FY-Q 

Lower  95% 
Confidence 

Observed  AA 
Rate 

Predicted  AA 
Rate 

Upper  95% 
Confidence 

Absolute 
Percent  Error 

2006-Q4 

0.5419 

0.5267* 

0.6217 

0.7014 

18.02% 

2007-Q1 

0.5316 

0.5265* 

0.6246 

0.7177 

18.64% 

2007-Q2 

0.5404 

0.5202* 

0.6414 

0.7424 

23.29% 

2007-Q3 

0.5339 

0.5120* 

0.6338 

0.7337 

23.78% 

2007-Q4 

0.4975 

0.5194 

0.5790 

0.6605 

11.49% 

2008-Q1 

0.5459 

0.5003* 

0.6526 

0.7593 

30.45% 

2008-Q2 

0.5241 

0.4302* 

0.6264 

0.7287 

45.62% 

2008-Q3 

0.4723 

0.3644* 

0.5835 

0.6947 

60.14% 

2008-Q4 

0.4205 

0.3373* 

0.5426 

0.6647 

60.89% 

*Observations  fall  outside  prediction  interval 

MAPE  =  32.48% 

Immediately  we  find  that  only  one  out  of  nine  observations  fell  within  the 
prediction  interval  generated  by  our  model.  The  failure  of  our  model  is  magnified  when 
we  consider  that  the  average  range  of  our  prediction  intervals  was  nearly  0.20  (i.e.,  20% 


in  terms  of  the  predicted  AA  rates).  Lastly,  our  model  produced  a  MAPE  of  32.48%. 
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Figure  9:  B-l  (ACC)  Sensitivity  Analysis 
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Figure  9  plainly  shows  that  B-ls  experienced  a  steady  and  dramatic  decline  in  AA 
rates  during  the  time  period  used  to  test  our  model,  but  we  were  unable  to  find  a 
definitive  reason  as  to  why  this  trend  occurred.  According  to  Air  Force  officials,  the 
decline  in  AA  rates  is  simply  a  sign  of  deterioration  on  individual  components,  not  an 
indication  of  a  specific  problem  (Rolfsen,  2008).  Additionally,  the  data  used  for  our 
variables  to  validate  the  model  fell  within  the  original  range  of  data  in  every  instance 
with  the  exception  of  the  data  used  for  cannibalization  hours  per  TAI,  which  used  four 
observations  below  the  original  range  of  data.  This  suggests  that  something  related  to  B- 
1  AA  rates  fundamentally  changed  during  the  final  nine  periods;  however,  we  were 
unable  to  capture  this  change  with  the  variables  for  which  we  had  data. 

KC-135  by  AETC  by  Quarter 

The  last  model  we  will  discuss  is  for  KC-135s  in  Air  Education  and  Training 
Command  (AETC).  As  shown  in  Table  20,  this  model  is  the  first  for  which  we  are  able 
to  use  total  EEIC  644  costs  (lagged  one  period)  as  an  independent  variable.  We  also  use 
average  aircraft  age  and  the  lagged  dependent  variable  as  regressors  in  this  model.  Again, 
we  must  include  the  lagged  dependent  variable  because  without  it  we  see  first  order 
autocorrelation  given  by  a  Durban-Watson  statistic  of  0.82  (significant  at  the  0.01  level). 
Furthennore,  the  adjusted  R2  drops  from  0.70286  to  0.5 1723  without  the  AA  rate  lagged 
one  period  included  in  the  model. 

Lastly,  we  verified  that  the  assumptions  of  the  random  error  term  were  satisfied, 
in  addition  to  checking  for  influential  data  points.  Like  the  first  three  models,  all 
assumptions  were  met. 
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Table  20:  KC-135  (AETC)  Explanatory  Model 


Summary  of  Fit 

R2 

0.72987 

Adjusted  R2 

0.70286 

Root  Mean  Square  Error 

0.04218 

Mean  of  Response 

0.73629 

Observations  (or  Sum  Wgts) 

34 

Analysis  of  Variance 

Source 

DF 

Sum  of  Squares  Mean  Square 

F  Ratio 

Model 

3 

0.14427 

0.04809 

27.0198 

Error 

30 

0.05339 

0.00178  Prob  >  F 

C.  Total 

33 

0.19767 

<.0001 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

VIF 

Intercept 

-0.15297 

0.11357 

-1.35 

0.1881 

T otalEEIC 644/T At  (Lag  1) 

4.473E-07 

2.272E-07 

1.97 

0.0583 

1.022 

Avg  Age 

0.01082 

0.00359 

3.01 

0.0052 

1.584 

lQLagAARate 

0.56119 

0.12435 

4.51 

<.0001 

1.575 

Once  more,  we  test  the  validity  of  our  model  by  including  the  final  nine  periods  of 

•  •  •  •  * 

independent  variable  data  and  running  the  model  in  JMP  .  Table  21  and  Figure  10 

provide  the  results  of  this  analysis. 


Table  21:  KC-135  (AETC)  Sensitivity  Analysis 


FY-Q 

Lower  95% 
Confidence 

Observed  AA 
Rate 

Predicted  AA 

Rate 

Upper  95% 
Confidence 

Absolute 
Percent  Error 

2006-Q4 

0.7718 

0.8492 

0.8643 

0.9569 

1.78% 

2007-Q1 

0.7770 

0.7876 

0.8702 

0.9633 

10.49% 

2007-Q2 

0.7163 

0.7464 

0.8097 

0.9032 

8.48% 

2007-Q3 

0.7033 

0.7576 

0.7977 

0.8921 

5.30% 

2007-Q4 

0.7239 

0.7715 

0.8186 

0.9132 

6.10% 

2008-Q1 

0.7301 

0.8416 

0.8251 

0.9200 

1.97% 

2008-Q2 

0.7517 

0.7724 

0.8494 

0.9471 

9.97% 

2008-Q3 

0.7174 

0.7834 

0.8167 

0.9160 

4.25% 

2008-Q4 

0.7427 

0.7310* 

0.8408 

0.9389 

15.01% 

*Observation  falls  outside  prediction  interval  MAPE  =  7.04% 

Although  eight  out  of  nine  observations  fall  within  the  prediction  interval,  we  can 
attribute  some  of  this  success  to  the  fact  that  our  model  produced  prediction  intervals 
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with  an  average  range  of  0. 19.  Such  a  large  range  when  forecasting  AA  rates  make  this 
model’s  usefulness  questionable.  Finally,  the  MAPE  for  this  model  is  7.04%. 


Observed  AA  Rate 
Predicted  AA  Rate 
Lower  95%  Prediction 
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Figure  10:  KC-135  (AETC)  Sensitivity  Analysis 
Summary  of  Remaining  Models 

Table  22  summarizes  the  remaining  models  to  include  the  adjusted  R“  values, 
variable  coefficients,  and  failed  assumptions  that  we  were  unable  to  avoid.  In  cases 
where  we  collected  data  on  MDS  that  operate  in  more  than  one  MAJCOM  (e.g.,  A-10,  F- 
15E),  we  attempted  to  develop  models  that  would  be  useful  in  predicting  AA  rates  for  the 
entire  fleet  of  aircraft  across  each  MAJCOM.  However,  due  to  the  apparent  differences 
in  predictive  variables,  we  were  unable  to  produce  useful  models  for  an  MDS  across  all 
of  its  MAJCOMs.  It  goes  without  saying,  therefore,  that  we  were  unable  to  develop 
useful  models  to  represent  more  than  one  MDS. 
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Table  22:  Summary  of  Models  Created  for  Remaining  MDS  and  MAJCOMs 


MDS 

MAJCOM 

<N 

< 

c* 

< 

a 

lQLag  AARate 

FlyHrs/  TAI 

Avg  Age 

EEIC644/  TAI 

(Lag  1) 

1,3  >5/7,9 ,0  Skill 

Level  Ratio 

(note  1) 

Total 

Maintainers/ 

TAI 

Other  Variables 

Failed 

Assumptions 

(see  codes  below) 

A-10 

PACAF 

0.55399 

34 

0.39325 

. 

. 

. 

-0.06561 

. 

0.04166 
(note  2) 

none 

A-10 

ACC 

0.52320 

34 

0.66465 

0.00117 

- 

- 

- 

- 

- 

none 

A-10 

USAFE 

0.40542 

34 

0.38250 

- 

- 

- 

0.05395 

- 

- 

none 

B-2 

ACC 

0.45791 

35 

- 

0.00145 

- 

- 

0.13274 

- 

- 

none 

B-52 

ACC 

0.30829 

34 

0.45696 

-0.00074 

- 

- 

- 

- 

- 

none 

F-15C/D 

ACC 

0.65512 

35 

- 

0.00189 

0.01215 

- 

- 

- 

- 

none 

F-15C/D 

AETC 

0.41891 

34 

0.45637 

1.0203E-07 

A 

(note  3) 

F-15C/D 

PACAF 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

F-15C/D 

USAFE 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

F-15E 

USAFE 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

F-16C/D 

ACC 

0.68421 

34 

0.49512 

- 

- 

- 

-0.04230 

0.00843 

- 

none 

F-16C/D 

PACAF 

0.63977 

33 

0.80798 

-0.02696 
(note  4) 

D 

(note  5) 

F-16C/D 

AETC 

0.41748 

35 

0.01518 

A 

(note  6) 

F-16C/D 

USAFE 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

KC-135 

PACAF 

0.49527 

34 

0.41306 

0.00179 

0.01975 

- 

-0.09711 

- 

- 

none 

KC-135 

USAFE 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

T-38 

AETC 

0.63190 

34 

0.64955 

0.00371 

D 

(note  7) 

T-38 

ACC 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

Failed  Assumptions :  (A)  Normality;  (B)  Constant  Variance;  (C)  Independence;  (D)  Influential  Data  Points 
Notes:  1.)  This  variable  represents  the  ratio  of  1-,  3-,  and  5-level  maintainers  to  7-,  9-,  and  0-level  maintainers . 

2.)  A  dummy  variable  to  represent  the  3rd  Quarter  of  each  fiscal  year  demonstrated  predictive  ability.  3.)  The 
model  failed  the  Shapiro-W ilk  test  for  nonnality  with  a  p-value  of 0.0429.  4.)  A  dummy  variable  to  represent  the 
4th  Quarter  of  each  fiscal  year  demonstrated  predictive  ability.  5.)  An  influential  data  point  was  removed  from 
the  model  because  it  significantly  changed  p-values.  6.)  The  model  failed  the  Shapiro-Wilktest  for  normality 
with  a  p-value  of  0.0332.  7.)  An  influential  data  point  was  allowed  to  remain  in  the  model  because  the  model 
was  not  changed  when  it  was  removed. 
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Further  Analysis 

Given  our  inability  to  develop  regression  models  that  show  a  definitive 
relationship  between  O&M  costs  and  AA  rates  using  our  preferred  strategy,  we  explore 
two  more  techniques. 

During  our  review  of  the  data,  we  notice  that  there  is  a  high  level  of  O&M  costs 
recorded  in  EEIC  644  in  the  fourth  quarter  of  each  year  for  almost  every  MDS  and 
MAJCOM  (when  compared  to  the  previous  quarters  of  the  same  fiscal  year).  An 
example  of  this  trend  is  shown  in  Figure  1 1  for  the  A-10  in  ACC.  This  occurrence  is  not 
surprising  since  units  often  have  to  spend  their  remaining  funds  at  a  higher  rate  at  the  end 
of  a  fiscal  year.  Accordingly,  this  phenomenon  should  make  it  possible  for  us  to  test  our 
hypothesis  that  increased  spending  leads  to  higher  AA  rates.  In  order  to  test  our  theory, 
we  use  dummy  variables  to  represent  each  quarter  of  the  fiscal  year  in  our  regression 
model.  If  there  is  a  statistically  significant  difference  between  one  of  the  quarters  and  the 
other  three,  we  will  see  this  in  the  fonn  of  a  low  p-value  and  a  significant  independent 
variable  in  our  model.  We  repeat  this  process  several  times  in  order  to  allow  each  quarter 
to  serve  as  the  baseline  and  include  various  combinations  of  the  dummy  variables 
representing  each  quarter  in  our  model.  However,  multiple  attempts  using  this  technique 
for  every  MDS  and  MAJCOM  failed  to  show  any  significant  difference  from  one  quarter 
to  the  next. 
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Fiscal  Year 


Figure  11:  A-10  (ACC)  EEIC  644  Costs  by  FY  Quarter 
For  our  next  attempt  at  modeling  AA  rates  using  O&M  costs,  we  take  a  different 

approach  and  use  two  of  the  variables  representing  non-availability  as  the  dependent 

variable  in  our  regression  models.  First,  we  use  the  NMCS  rate  as  the  dependent  variable 

in  order  to  see  if  we  can  model  the  percentage  of  aircraft  that  are  not  mission  capable  for 

supply  reasons.  Second,  we  use  the  NMCM  rate  as  the  dependent  variable  and  attempt  to 

model  the  percentage  of  aircraft  that  are  not  mission  capable  for  maintenance  reasons.  If 

we  are  able  to  produce  predictive  models  that  explain  either  NMCS  or  NMCM,  we  will 

be  able  to  extract  results  that  are  nearly  as  useful  as  modeling  AA  rates  directly. 

However,  using  these  metrics  as  dependent  variables  does  not  produce  better  models. 


Chapter  Summary 

In  Chapter  IV,  we  explained  the  challenges  presented  by  our  data  and  the 
techniques  we  used  to  overcome  these  difficulties.  Next,  we  detailed  the  results  of  our 
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four  best  explanatory  models  and  summarized  the  remaining  models.  Finally,  we 
explored  two  other  techniques  for  examining  our  data,  but  with  only  marginal  success. 

In  the  next  chapter,  we  use  our  findings  to  address  our  research  questions.  Then, 
we  discuss  policy  implications  and  the  strengths  and  limitations  of  our  study  as  well  as 
opportunities  for  further  research. 
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V:  Conclusions 


In  this  chapter,  we  provide  answers  to  our  research  questions  based  on  the  results 
of  our  study.  Next,  we  discuss  potential  policy  implications.  Finally,  we  highlight  the 
strengths  and  limitations  of  our  study  while  suggesting  areas  for  further  research. 

Research  Questions 

1.  What  variables  are  significant  predictors  of  AA  rates? 

Of  the  16  models  we  developed,  we  find  that  the  dependent  variable  lagged  one 
period  is  a  significant  predictor  in  13  of  our  models.  Admittedly,  this  variable  was 
included  primarily  to  help  mitigate  the  negative  effects  of  autocorrelation  in  the  residuals, 
and  its  explanatory  contribution  is  of  limited  usefulness  to  decision  makers.  By 
definition,  we  must  have  the  current  period’s  AA  rate  in  order  to  forecast  the  next 
period’s  AA  rate.  This  requirement  makes  it  very  difficult  for  the  models  to  be  useful 
beyond  one  period  into  the  future. 

Next,  we  find  that  the  flying  hours  per  TAI  variable  is  a  predictive  independent 
variable  in  seven  of  our  models.  For  all  but  one,  the  coefficient  is  positive  which 
suggests  that  flying  aircraft  more  often  increases  the  AA  rate.  The  coefficient  is  only 
negative  for  the  B-52  by  ACC  by  quarter  model. 

Finally,  our  results  show  that  the  variable  representing  the  ratio  of  1-,  3-,  and  5- 
level  maintainers  to  7-,  9-,  and  0-level  maintainers  is  predictive  in  five  of  our  models; 
however,  the  coefficients  are  mixed.  Three  models  show  negative  coefficients,  which 
imply  that  larger  numbers  of  low  skilled  maintainers  decrease  the  effectiveness  of  the 
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maintenance  being  performed,  resulting  in  lower  AA  rates.  The  other  two  models  result 
in  positive  coefficients,  which  imply  the  opposite  effect.  Intuitively,  this  makes  less 
sense.  We  hypothesize  that  where  positive  coefficients  are  found,  this  variable  is 
reflecting  some  other  effect  such  as  larger  numbers  of  maintainers  in  general. 

2.  Are  A  A  rates  influenced  by  changes  in  O&M  spending? 

From  the  analysis  we  performed,  we  are  unable  to  show  that  AA  rates  are 
significantly  influenced  by  changes  in  the  amount  of  O&M  spending.  Using  total  EEIC 
644  costs  as  our  variable  to  represent  O&M  spending,  we  find  that  it  is  predictive  in  only 
2  of  our  16  models  (12.5%).  Both  the  KC-135  by  AETC  by  quarter  model  and  the  F- 
15  C/D  by  AETC  by  quarter  model  found  total  EEIC  644  costs  to  be  useful  in  predicting 
A  A  rates. 

3.  Do  the  AA  rates  of  some  weapon  systems  respond  to  changes  in  O&M  costs 

more  than  others? 

Our  models  indicated  that  AA  rates  of  only  the  KC-135  in  AETC  and  the  F- 
15C/D  in  AETC  responded  to  changes  in  O&M  spending.  Given  our  limited  findings,  we 
are  unable  to  determine  if  the  AA  rates  of  some  weapon  systems  respond  to  changes  in 
O&M  spending  more  than  others. 

4.  Can  a  single  model  be  developed  to  represent  multiple  MDS? 

Despite  repeated  attempts,  we  were  unable  to  develop  a  model  that  would 

represent  multiple  MDS  or  multiple  MAJCOMs  for  a  single  MDS.  The  models  we 
created  are  specific  to  a  particular  MDS  and  MAJCOM. 

5.  Can  the  models  produced  by  this  research  be  used  as  an  effective  decision  tool 

for  the  Centralized  Asset  Management  (CAM)  office? 
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Given  our  lack  of  findings  with  respect  to  O&M  costs  and  AA,  we  do  not  believe 
that  the  CAM  office  would  find  our  models  useful  for  allocating  resources.  With  more 
research,  we  hope  that  models  will  be  developed  that  shed  light  on  the  relationship 
between  O&M  costs  and  AA  rates. 

Policy  Implications 

As  we  discussed  in  Chapter  II,  CAM  is  significantly  changing  the  way  Air  Force 
maintenance  organizations  acquire  and  pay  for  parts  and  supplies.  Instead  of  purchasing 
items  at  the  base  level,  CAM  now  centrally  manages  the  process  for  all  active  duty  units. 
Although  we  previously  focused  on  the  many  positive  attributes  of  the  new  process,  we 
must  consider  the  potential  negative  consequences  of  centralization.  Before  CAM  was 
implemented,  base  level  maintenance  organizations  were  incentivized  to  be  fiscally 
responsible.  They  were  constrained  by  their  base  level  budget,  which  encouraged  them 
to  be  cost  effective  and  repair  parts  which  could  be  redeemed  for  credits  to  pay  for  other 
items.  Now  that  funds  are  centrally  managed,  the  Air  Force  must  ensure  that  base  level 
maintenance  organizations  continue  to  operate  responsibly.  If  base  level  organizations  no 
longer  feel  constrained  by  their  local  budgets,  they  may  begin  to  operate  in  a  less  cost 
effective  manner  resulting  in  higher  overall  costs  to  the  Air  Force. 

Strengths,  Limitations,  and  Further  Research 

Although  our  research  failed  to  draw  a  definitive  link  between  O&M  costs  and 
AA  rates,  the  data  we  collected  and  the  variables  we  used  in  our  study  were  reliable.  Our 
data  was  extracted  directly  from  official  Air  Force  databases  and  we  used  extreme  care 
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when  compiling  our  database.  Furthennore,  we  carefully  analyzed  our  variables  and 
incorporated  them  in  our  models  in  a  fashion  that  would  not  result  in  disingenuous 
findings. 

We  chose  total  costs  recorded  in  EEIC  644  to  represent  O&M  costs  because  we 
were  able  to  show  a  clear  link  between  the  money  spent  and  the  aircraft  type  supported. 
In  hindsight,  this  may  have  been  a  weakness  in  our  analysis.  The  CAM  office  manages  a 
large  portfolio  of  funds  for  the  Air  Force.  Figure  12  shows  the  funding  posture  for 
FY2010,  which  totals  $12,363.1  million.  Budgeted  at  $2,147.8  million,  EEIC  644 
accounts  for  nearly  half  of  the  flying  hour  program;  however,  we  can  plainly  see  that  the 
majority  of  O&M  costs  managed  by  CAM  are  not  represented  by  our  variable.  We 
suggest  that  further  research  be  done  to  include  Depot  Purchased  Equipment 
Maintenance  (EEICs  540,  541,  542,  543,  544,  545,  546,  548,  and  549),  Sustaining 
Engineering  (EEIC  583),  Technical  Orders  (EEIC  594),  and  Flying  Hour  Program  costs 


(EEICs  605,  609,  61952,  69302,  and  699  in  addition  to  EEIC  644). 
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Figure  12:  CAM  FY2010  Funding  Posture  (in  millions,  BY10$) 
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We  selected  our  range  of  data  to  include  FY1998  -  2008  because  we  were  able  to 
ensure  consistent  accounting  of  EEIC  644  costs  during  that  timeframe.  Given  several 
more  years  of  post-CAM  data  (FY2008  and  beyond),  we  feel  that  this  study  may  produce 
entirely  different  results.  As  we  discussed  in  Chapter  II,  the  resource  allocation  process 
prior  to  CAM  was  not  designed  to  manage  weapon  systems  from  a  performance-based 
perspective.  Now  that  the  Air  Force  is  operating  with  CAM  and  a  philosophy  of 
“performance-based  logistics,”  we  hypothesize  that  a  study  done  in  the  future  would 
yield  more  useful  and  interesting  results. 
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Appendix  A:  Summary  of  Results  for  All  Models 


Summary  of  Regression  Coefficients  and  Adjusted  R2  Values 


MDS 

MAJCOM 

Adj  RA2 

S 

lQLag  AARate 

FlyHrs/  TAI 

Avg  Age 

EEIC644/  TAI 

(Lag  1) 

CANNhrs/  TAI 

2 

m 

1 

T-  _ 

H 

1,3,5/  7,9,0  Skill 

Level  Ratio 

(note  1) 

Total  Maintainers/ 

TAI 

Other  Variables 

Failed  Assumptions 

(see  codes  below) 

A-10 

PACAF 

0.55399 

34 

0.39325 

. 

. 

. 

. 

. 

-0.06561 

. 

0.04166 
(note  2) 

none 

A-10 

ACC 

0.52320 

34 

0.66465 

0.00117 

- 

- 

- 

- 

- 

- 

- 

none 

A-10 

USAFE 

0.40542 

34 

0.38250 

- 

- 

- 

- 

- 

0.05395 

- 

- 

none 

B-l 

ACC 

0.76779 

34 

0.61311 

0.00177 

- 

- 

-0.00139 

-0.0105 

- 

- 

- 

none 

B-2 

ACC 

0.45791 

35 

- 

0.00145 

- 

- 

- 

- 

0.13274 

- 

- 

none 

B-52 

ACC 

0.30829 

34 

0.45696 

-0.00074 

- 

- 

- 

- 

- 

- 

- 

none 

F-15C/D 

ACC 

0.65512 

35 

- 

0.00189 

0.01215 

- 

- 

- 

- 

- 

- 

none 

F-15C/D 

AETC 

0.41891 

34 

0.45637 

1.0203E-07 

A 

(note  3) 

F-15C/D 

PACAF 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

F-15C/D 

USAFE 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

F-15E 

ACC 

0.80918 

34 

0.38525 

- 

- 

- 

-0.00105 

0.01015 

- 

- 

- 

none 

F-15E 

USAFE 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

F-16C/D 

ACC 

0.68421 

34 

0.49512 

- 

- 

- 

- 

- 

-0.04230 

0.00843 

- 

none 

F-16C/D 

PACAF 

0.63977 

33 

0.80798 

-0.02696 
(note  4) 

D 

(note  5) 

F-16C/D 

AETC 

0.41748 

35 

0.01518 

A 

(note  6) 

F-16C/D 

USAFE 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

KC-135 

AMC 

0.86945 

34 

0.67384 

0.000612 

- 

- 

- 

- 

- 

- 

- 

none 

KC-135 

AETC 

0.70286 

34 

0.56119 

- 

0.01082 

4.473  IE-07 

- 

- 

- 

- 

- 

none 

KC-135 

PACAF 

0.49527 

34 

0.41306 

0.00179 

0.01975 

- 

- 

- 

-0.09711 

- 

- 

none 

KC-135 

USAFE 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

T-38 

AETC 

0.63190 

34 

0.64955 

0.00371 

D 

(note  7) 

T-38 

ACC 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

Failed  Assumptions:  (A)  Normality;  (B)  Constant  Variance;  (C)  Independence;  (D)  Influential  Data  Points 

Notes :  1 .)  This  variable  represents  the  ratio  of  1-,  3-,  and  5-level maintainers  to  7-,  9-,  and  0-level  maintainers.  2.)  A  dummy 
variable  to  represent  the  3rd  Quarter  of  each  fiscal  year  demonstrated  predictive  ability.  3.)  The  model  failed  the  Shapiro-W  ilk 
test  for  normality  with  a  p-value  of  0.0429.  4.)  A  dummy  variable  to  represent  the  4th  Quarter  of  each  fiscal  year  demonstrated 
predictive  ability.  5.)  An  influential  data  point  was  removed  from  the  model  because  it  significantly  changed  p-values.  6.)  The 
model  failed  the  Shapiro-W  ilk  test  for  nonnality  with  a  p-value  of  0.0332.  7.)  An  influential  data  point  was  allowed  to  remain  in 
the  model  because  the  model  was  not  changed  when  it  was  removed. 
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Appendix  B:  Sample  of  OLS  Regression  Diagnostic  Tests 


KC-135  by  AMC  by  Quarter  Model 

First,  we  show  a  histogram  of  studentized  residuals  followed  by  the  Shapiro-Wilk 
Test  for  normality.  The  null  hypothesis  is  that  the  residuals  come  from  a  nonnal 
distribution;  small  p-values  reject  the  null  hypothesis.  Alpha  is  0.05  for  all  tests. 


Shapiro-Wilk  Test 


w 

Prob<W 

0.9768 

0.6704 

Next,  we  show  the  residual  by  predicted  plot  and  the  Breusch-Pagan  test  for 
constant  variance  of  the  residuals.  The  null  hypothesis  is  that  the  residuals  exhibit 
constant  variance;  small  p-values  reject  the  null  hypothesis. 


0.06 


n - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 

0.45  0.50  0.55  0.60  0.65  0.70 

AA  Rate  Predicted 


Breusch-Pagan  Test 


p-value 


0.678 
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Third,  we  show  the  residual  by  row  plot  and  the  Durbin- Watson  test  for 
autocorrelation.  The  null  hypothesis  is  that  the  residuals  are  not  serially  correlated; 
small  p-values  reject  the  null  hypothesis. 


0.06 -i - — - 

§  0.02-  ■■■  .  '  ■  "  . 

-g  ■  _ '  _  _ 

v)  ”■  =— "  :  “ 

£  -0.02-  .  . 

-°06-| — — i — i — i — i — i — i — 

0  5  10  15  20  25  30  35  40  45 

Row  Number 


Durbin- Watson  Test 


Durbin- 

Watson 

Number  of 
Obs. 

Autocorrelation 

Prob<DW 

1.6044 

34 

0.1747 

0.0704 

Last,  we  show  the  Cook’s  Distance  plot  for  influential  data  points.  We  use  a 
value  of  0.5  as  our  cutoff  for  points  that  are  influential  and  require  additional  inspection. 
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Appendix  C:  Range  of  Independent  Variable  Data  Used  to  Construct  Models 


KC-135  by  AMC  by  Quarter  Model 


Variable 

Min 

Max 

lQLagAARate 

0.4637 

0.7091 

FlyHrs/TAI 

67.57 

184.86 

F-15E  by  ACC  by  Quarter  Model 


Variable 

Min 

Max 

lQLagAARate 

0.5854 

0.7238 

CANNhrs/TAI 

23.35 

101.21 

1,3,5  SkillLvl/TAI 

14.45 

16.68 

B-l  by  ACC  by  Quarter 


Variable 

Min 

Max 

lQLagAARate 

0.3797 

0.6205 

CANNhrs/TAI 

46.94 

141.59 

1,3,5  SkillLvl/TAI 

29.27 

41.36 

FlyHrs/TAI 

52.89 

106.99 

KC-135  by  AETC  by  Quarter 


Variable 

Min 

Max 

lQLagAARate 

0.5877 

0.8476 

TotalEEIC644/TAI  (Lag  1) 

44,920 

188,551 

Avg  Age 

35.22 

43.67 
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